Awesome Software Engineering for Machine Learning
![Awesome](https://awesome.re/badge-flat2.svg)
Software Engineering for Machine Learning are techniques and guidelines
for building ML applications that do not concern the core ML problem β
e.g.Β the development of new algorithms β but rather the surrounding
activities like data ingestion, coding, testing, versioning, deployment,
quality control, and team collaboration. Good software engineering
practices enhance development, deployment and maintenance of production
level applications using machine learning components.
β Must-read
π Scientific publication
Based on this literature, we compiled a survey on the adoption of software
engineering practices for applications with machine learning components.
Feel free to
take and share the survey and
to read more!
Contents
Broad Overviews
These resources cover all aspects. -
AI Engineering: 11 Foundational Practices
β -
Best Practices for Machine Learning Applications
-
Engineering Best Practices for Machine Learning
β -
Hidden Technical Debt in Machine Learning Systems
πβ -
Rules of Machine Learning: Best Practices for ML Engineering
β -
Software Engineering for Machine Learning: A Case Study
πβ
Data Management
How to manage the data sets you use in machine learning.
Model Training
How to organize your model training experiments.
Deployment and Operation
How to deploy and operate your models in a production environment.
Social Aspects
How to organize teams and projects to ensure effective collaboration and
accountability.
Governance
Tooling can make your life easier.
We only share open source tools, or commercial platforms that offer
substantial free packages for research.
-
Airflow - Programmatically
author, schedule and monitor workflows.
-
Alibi Detect -
Python library focused on outlier, adversarial and drift detection.
-
Archai - Neural
architecture search.
-
Data Version Control (DVC) - DVC is a
data and ML experiments management tool.
-
Facets Overview / Facets Dive
- Robust visualizations to aid in understanding machine learning
datasets.
-
FairLearn - A toolkit to
assess and improve the fairness of machine learning models.
-
Git Large File System (LFS) -
Replaces large files such as datasets with text pointers inside Git.
-
Great Expectations
- Data validation and testing with integration in pipelines.
-
HParams - A
thoughtful approach to configuration management for machine learning
projects.
-
Kubeflow - A platform for data
scientists who want to build and experiment with ML pipelines.
-
Label Studio -
A multi-type data labeling and annotation tool with standardized output
format.
-
LiFT - Linkedin fairness
toolkit.
-
MLFlow - Manage the ML lifecycle,
including experimentation, deployment, and a central model registry.
-
Model Card Toolkit
- Streamlines and automates the generation of model cards; for model
documentation.
-
Neptune.ai - Experiment tracking tool
bringing organization and collaboration to data science projects.
-
Neuraxle -
Sklearn-like framework for hyperparameter tuning and AutoML in deep
learning projects.
-
OpenML - An inclusive movement to
build an open, organized, online ecosystem for machine learning.
-
PyTorch Lightning
- The lightweight PyTorch wrapper for high-performance AI research.
Scale your models, not the boilerplate.
-
Robustness Metrics
- Lightweight modules to evaluate the robustness of classification
models.
-
Seldon Core - An
MLOps framework to package, deploy, monitor and manage thousands of
production machine learning models on Kubernetes.
-
Spark Machine Learning -
Sparkβs ML library consisting of common learning algorithms and
utilities.
-
TensorBoard -
TensorFlowβs Visualization Toolkit.
-
Tensorflow Extended (TFX)
- An end-to-end platform for deploying production ML pipelines.
-
Tensorflow Data Validation (TFDV)
- Library for exploring and validating machine learning data. Similar to
Great Expectations, but for Tensorflow data.
-
Weights & Biases - Experiment
tracking, model optimization, and dataset versioning.
Contribute
Contributions welcomed! Read the
contribution guidelines first