rhetenor/awesome-awesomeness
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update lists

Browse Source
This commit is contained in:
Jonas Zeunert
2025-07-18 22:22:32 +02:00
parent 55bed3b4a1
commit 5916c5c074
3078 changed files with 331679 additions and 357255 deletions
Download Patch File Download Diff File Expand all files Collapse all files

930
html/mlops.html
View File

@@ -1,930 +0,0 @@
<h1 id="awesome-mlops-awesome">Awesome MLOps <a
href="https://github.com/sindresorhus/awesome"><img
src="https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg"
alt="Awesome" /></a></h1>
<p>A curated list of awesome MLOps tools.</p>
<p>Inspired by <a
href="https://github.com/vinta/awesome-python">awesome-python</a>.</p>
<ul>
<li><a href="#awesome-mlops">Awesome MLOps</a>
<ul>
<li><a href="#automl">AutoML</a></li>
<li><a href="#cicd-for-machine-learning">CI/CD for Machine
Learning</a></li>
<li><a href="#cron-job-monitoring">Cron Job Monitoring</a></li>
<li><a href="#data-catalog">Data Catalog</a></li>
<li><a href="#data-enrichment">Data Enrichment</a></li>
<li><a href="#data-exploration">Data Exploration</a></li>
<li><a href="#data-management">Data Management</a></li>
<li><a href="#data-processing">Data Processing</a></li>
<li><a href="#data-validation">Data Validation</a></li>
<li><a href="#data-visualization">Data Visualization</a></li>
<li><a href="#drift-detection">Drift Detection</a></li>
<li><a href="#feature-engineering">Feature Engineering</a></li>
<li><a href="#feature-store">Feature Store</a></li>
<li><a href="#hyperparameter-tuning">Hyperparameter Tuning</a></li>
<li><a href="#knowledge-sharing">Knowledge Sharing</a></li>
<li><a href="#machine-learning-platform">Machine Learning
Platform</a></li>
<li><a href="#model-fairness-and-privacy">Model Fairness and
Privacy</a></li>
<li><a href="#model-interpretability">Model Interpretability</a></li>
<li><a href="#model-lifecycle">Model Lifecycle</a></li>
<li><a href="#model-serving">Model Serving</a></li>
<li><a href="#model-testing--validation">Model Testing &amp;
Validation</a></li>
<li><a href="#optimization-tools">Optimization Tools</a></li>
<li><a href="#simplification-tools">Simplification Tools</a></li>
<li><a href="#visual-analysis-and-debugging">Visual Analysis and
Debugging</a></li>
<li><a href="#workflow-tools">Workflow Tools</a></li>
</ul></li>
<li><a href="#resources">Resources</a>
<ul>
<li><a href="#articles">Articles</a></li>
<li><a href="#books">Books</a></li>
<li><a href="#events">Events</a></li>
<li><a href="#other-lists">Other Lists</a></li>
<li><a href="#podcasts">Podcasts</a></li>
<li><a href="#slack">Slack</a></li>
<li><a href="#websites">Websites</a></li>
</ul></li>
<li><a href="#contributing">Contributing</a></li>
</ul>
<hr />
<h2 id="automl">AutoML</h2>
<p><em>Tools for performing AutoML.</em></p>
<ul>
<li><a href="https://github.com/awslabs/autogluon">AutoGluon</a> -
Automated machine learning for image, text, tabular, time-series, and
multi-modal data.</li>
<li><a href="https://github.com/keras-team/autokeras">AutoKeras</a> -
AutoKeras goal is to make machine learning accessible for everyone.</li>
<li><a href="https://github.com/automl/Auto-PyTorch">AutoPyTorch</a> -
Automatic architecture search and hyperparameter optimization for
PyTorch.</li>
<li><a href="https://github.com/automl/auto-sklearn">AutoSKLearn</a> -
Automated machine learning toolkit and a drop-in replacement for a
scikit-learn estimator.</li>
<li><a href="https://github.com/alteryx/evalml">EvalML</a> - A library
that builds, optimizes, and evaluates ML pipelines using domain-specific
functions.</li>
<li><a href="https://github.com/microsoft/FLAML">FLAML</a> - Finds
accurate ML models automatically, efficiently and economically.</li>
<li><a href="https://h2o.ai/platform/h2o-automl">H2O AutoML</a> -
Automates ML workflow, which includes automatic training and tuning of
models.</li>
<li><a href="https://github.com/mindsdb/mindsdb">MindsDB</a> - AI layer
for databases that allows you to effortlessly develop, train and deploy
ML models.</li>
<li><a href="https://github.com/AxeldeRomblay/MLBox">MLBox</a> - MLBox
is a powerful Automated Machine Learning python library.</li>
<li><a href="https://github.com/google/model_search">Model Search</a> -
Framework that implements AutoML algorithms for model architecture
search at scale.</li>
<li><a href="https://github.com/microsoft/nni">NNI</a> - An open source
AutoML toolkit for automate machine learning lifecycle.</li>
</ul>
<h2 id="cicd-for-machine-learning">CI/CD for Machine Learning</h2>
<p><em>Tools for performing CI/CD for Machine Learning.</em></p>
<ul>
<li><a href="https://github.com/allegroai/clearml">ClearML</a> -
Auto-Magical CI/CD to streamline your ML workflow.</li>
<li><a href="https://github.com/iterative/cml">CML</a> - Open-source
library for implementing CI/CD in machine learning projects.</li>
</ul>
<h2 id="cron-job-monitoring">Cron Job Monitoring</h2>
<p><em>Tools for monitoring cron jobs (recurring jobs).</em></p>
<ul>
<li><a href="https://cronitor.io/cron-job-monitoring">Cronitor</a> -
Monitor any cron job or scheduled task.</li>
<li><a href="https://healthchecks.io/">HealthchecksIO</a> - Simple and
effective cron job monitoring.</li>
</ul>
<h2 id="data-catalog">Data Catalog</h2>
<p><em>Tools for data cataloging.</em></p>
<ul>
<li><a href="https://www.amundsen.io/">Amundsen</a> - Data discovery and
metadata engine for improving the productivity when interacting with
data.</li>
<li><a href="https://atlas.apache.org">Apache Atlas</a> - Provides open
metadata management and governance capabilities to build a data
catalog.</li>
<li><a href="https://github.com/ckan/ckan">CKAN</a> - Open-source DMS
(data management system) for powering data hubs and data portals.</li>
<li><a href="https://github.com/linkedin/datahub">DataHub</a> -
LinkedIn’s generalized metadata search &amp; discovery tool.</li>
<li><a href="https://github.com/magda-io/magda">Magda</a> - A federated,
open-source data catalog for all your big data and small data.</li>
<li><a href="https://github.com/Netflix/metacat">Metacat</a> - Unified
metadata exploration API service for Hive, RDS, Teradata, Redshift, S3
and Cassandra.</li>
<li><a href="https://open-metadata.org/">OpenMetadata</a> - A Single
place to discover, collaborate and get your data right.</li>
</ul>
<h2 id="data-enrichment">Data Enrichment</h2>
<p><em>Tools and libraries for data enrichment.</em></p>
<ul>
<li><a href="https://github.com/snorkel-team/snorkel">Snorkel</a> - A
system for quickly generating training data with weak supervision.</li>
<li><a href="https://github.com/upgini/upgini">Upgini</a> - Enriches
training datasets with features from public and community shared data
sources.</li>
</ul>
<h2 id="data-exploration">Data Exploration</h2>
<p><em>Tools for performing data exploration.</em></p>
<ul>
<li><a href="https://zeppelin.apache.org/">Apache Zeppelin</a> - Enables
data-driven, interactive data analytics and collaborative
documents.</li>
<li><a href="https://github.com/tkrabel/bamboolib">BambooLib</a> - An
intuitive GUI for Pandas DataFrames.</li>
<li><a href="https://github.com/sfu-db/dataprep">DataPrep</a> - Collect,
clean and visualize your data in Python.</li>
<li><a href="https://colab.research.google.com">Google Colab</a> -
Hosted Jupyter notebook service that requires no setup to use.</li>
<li><a href="https://jupyter.org/">Jupyter Notebook</a> - Web-based
notebook environment for interactive computing.</li>
<li><a href="https://jupyterlab.readthedocs.io">JupyterLab</a> - The
next-generation user interface for Project Jupyter.</li>
<li><a href="https://github.com/mwouts/jupytext">Jupytext</a> - Jupyter
Notebooks as Markdown Documents, Julia, Python or R scripts.</li>
<li><a href="https://github.com/ydataai/pandas-profiling">Pandas
Profiling</a> - Create HTML profiling reports from pandas DataFrame
objects.</li>
<li><a href="https://polynote.org/">Polynote</a> - The polyglot notebook
with first-class Scala support.</li>
</ul>
<h2 id="data-management">Data Management</h2>
<p><em>Tools for performing data management.</em></p>
<ul>
<li><a href="https://www.arrikto.com/">Arrikto</a> - Dead simple, ultra
fast storage for the hybrid Kubernetes world.</li>
<li><a href="https://github.com/BlazingDB/blazingsql">BlazingSQL</a> - A
lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS
cuDF.</li>
<li><a href="https://github.com/delta-io/delta">Delta Lake</a> - Storage
layer that brings scalable, ACID transactions to Apache Spark and other
engines.</li>
<li><a href="https://github.com/dolthub/dolt">Dolt</a> - SQL database
that you can fork, clone, branch, merge, push and pull just like a git
repository.</li>
<li><a href="https://github.com/kevin-hanselman/dud">Dud</a> - A
lightweight CLI tool for versioning data alongside source code and
building data pipelines.</li>
<li><a href="https://dvc.org/">DVC</a> - Management and versioning of
datasets and machine learning models.</li>
<li><a href="https://git-lfs.github.com">Git LFS</a> - An open source
Git extension for versioning large files.</li>
<li><a href="https://github.com/activeloopai/Hub">Hub</a> - A dataset
format for creating, storing, and collaborating on AI datasets of any
size.</li>
<li><a href="https://github.com/intake/intake">Intake</a> - A
lightweight set of tools for loading and sharing data in data science
projects.</li>
<li><a href="https://github.com/treeverse/lakeFS">lakeFS</a> -
Repeatable, atomic and versioned data lake on top of object
storage.</li>
<li><a href="https://github.com/MarquezProject/marquez">Marquez</a> -
Collect, aggregate, and visualize a data ecosystem’s metadata.</li>
<li><a href="https://github.com/milvus-io/milvus/">Milvus</a> - An open
source embedding vector similarity search engine powered by Faiss,
NMSLIB and Annoy.</li>
<li><a href="https://www.pinecone.io">Pinecone</a> - Managed and
distributed vector similarity search used with a lightweight SDK.</li>
<li><a href="https://github.com/qdrant/qdrant">Qdrant</a> - An open
source vector similarity search engine with extended filtering
support.</li>
<li><a href="https://github.com/quiltdata/quilt">Quilt</a> - A
self-organizing data hub with S3 support.</li>
</ul>
<h2 id="data-processing">Data Processing</h2>
<p><em>Tools related to data processing and data pipelines.</em></p>
<ul>
<li><a href="https://airflow.apache.org/">Airflow</a> - Platform to
programmatically author, schedule, and monitor workflows.</li>
<li><a href="https://github.com/azkaban/azkaban">Azkaban</a> - Batch
workflow job scheduler created at LinkedIn to run Hadoop jobs.</li>
<li><a href="https://github.com/dagster-io/dagster">Dagster</a> - A data
orchestrator for machine learning, analytics, and ETL.</li>
<li><a href="https://hadoop.apache.org/">Hadoop</a> - Framework that
allows for the distributed processing of large data sets across
clusters.</li>
<li><a href="https://github.com/OpenRefine/OpenRefine">OpenRefine</a> -
Power tool for working with messy data and improving it.</li>
<li><a href="https://spark.apache.org/">Spark</a> - Unified analytics
engine for large-scale data processing.</li>
</ul>
<h2 id="data-validation">Data Validation</h2>
<p><em>Tools related to data validation.</em></p>
<ul>
<li><a href="https://github.com/pyeve/cerberus">Cerberus</a> -
Lightweight, extensible data validation library for Python.</li>
<li><a href="https://github.com/cleanlab/cleanlab">Cleanlab</a> - Python
library for data-centric AI and machine learning with messy, real-world
data and labels.</li>
<li><a href="https://greatexpectations.io">Great Expectations</a> - A
Python data validation framework that allows to test your data against
datasets.</li>
<li><a href="https://json-schema.org/">JSON Schema</a> - A vocabulary
that allows you to annotate and validate JSON documents.</li>
<li><a href="https://github.com/tensorflow/data-validation">TFDV</a> -
An library for exploring and validating machine learning data.</li>
</ul>
<h2 id="data-visualization">Data Visualization</h2>
<p><em>Tools for data visualization, reports and dashboards.</em></p>
<ul>
<li><a href="https://count.co">Count</a> - SQL/drag-and-drop querying
and visualisation tool based on notebooks.</li>
<li><a href="https://github.com/plotly/dash">Dash</a> - Analytical Web
Apps for Python, R, Julia, and Jupyter.</li>
<li><a href="https://datastudio.google.com">Data Studio</a> - Reporting
solution for power users who want to go beyond the data and dashboards
of GA.</li>
<li><a href="https://github.com/PAIR-code/facets">Facets</a> -
Visualizations for understanding and analyzing machine learning
datasets.</li>
<li><a href="https://grafana.com/grafana/">Grafana</a> - Multi-platform
open source analytics and interactive visualization web
application.</li>
<li><a href="https://github.com/lux-org/lux">Lux</a> - Fast and easy
data exploration by automating the visualization and data analysis
process.</li>
<li><a href="https://www.metabase.com/">Metabase</a> - The simplest,
fastest way to get business intelligence and analytics to everyone.</li>
<li><a href="https://redash.io/">Redash</a> - Connect to any data
source, easily visualize, dashboard and share your data.</li>
<li><a href="https://github.com/CloudOrc/SolidUI">SolidUI</a> -
AI-generated visualization prototyping and editing platform, support 2D
and 3D models.</li>
<li><a href="https://superset.incubator.apache.org/">Superset</a> -
Modern, enterprise-ready business intelligence web application.</li>
<li><a href="https://www.tableau.com">Tableau</a> - Powerful and fastest
growing data visualization tool used in the business intelligence
industry.</li>
</ul>
<h2 id="drift-detection">Drift Detection</h2>
<p><em>Tools and libraries related to drift detection.</em></p>
<ul>
<li><a href="https://github.com/SeldonIO/alibi-detect">Alibi Detect</a>
- An open source Python library focused on outlier, adversarial and
drift detection.</li>
<li><a href="https://github.com/IFCA/frouros">Frouros</a> - An open
source Python library for drift detection in machine learning
systems.</li>
<li><a href="https://github.com/torchdrift/torchdrift/">TorchDrift</a> -
A data and concept drift library for PyTorch.</li>
</ul>
<h2 id="feature-engineering">Feature Engineering</h2>
<p><em>Tools and libraries related to feature engineering.</em></p>
<ul>
<li><a href="https://github.com/feature-engine/feature_engine">Feature
Engine</a> - Feature engineering package with SKlearn like
functionality.</li>
<li><a href="https://github.com/alteryx/featuretools">Featuretools</a> -
Python library for automated feature engineering.</li>
<li><a href="https://github.com/blue-yonder/tsfresh">TSFresh</a> -
Python library for automatic extraction of relevant features from time
series.</li>
</ul>
<h2 id="feature-store">Feature Store</h2>
<p><em>Feature store tools for data serving.</em></p>
<ul>
<li><a href="https://github.com/quintoandar/butterfree">Butterfree</a> -
A tool for building feature stores. Transform your raw data into
beautiful features.</li>
<li><a href="https://github.com/bytehub-ai/bytehub">ByteHub</a> - An
easy-to-use feature store. Optimized for time-series data.</li>
<li><a href="https://feast.dev/">Feast</a> - End-to-end open source
feature store for machine learning.</li>
<li><a href="https://github.com/linkedin/feathr">Feathr</a> - An
enterprise-grade, high performance feature store.</li>
<li><a href="https://github.com/featureform/featureform">Featureform</a>
- A Virtual Feature Store. Turn your existing data infrastructure into a
feature store.</li>
<li><a href="https://www.tecton.ai/">Tecton</a> - A fully-managed
feature platform built to orchestrate the complete lifecycle of
features.</li>
</ul>
<h2 id="hyperparameter-tuning">Hyperparameter Tuning</h2>
<p><em>Tools and libraries to perform hyperparameter tuning.</em></p>
<ul>
<li><a href="https://github.com/tobegit3hub/advisor">Advisor</a> -
Open-source implementation of Google Vizier for hyper parameters
tuning.</li>
<li><a href="https://github.com/maxpumperla/hyperas">Hyperas</a> - A
very simple wrapper for convenient hyperparameter optimization.</li>
<li><a href="https://github.com/hyperopt/hyperopt">Hyperopt</a> -
Distributed Asynchronous Hyperparameter Optimization in Python.</li>
<li><a href="https://github.com/kubeflow/katib">Katib</a> -
Kubernetes-based system for hyperparameter tuning and neural
architecture search.</li>
<li><a href="https://github.com/keras-team/keras-tuner">KerasTuner</a> -
Easy-to-use, scalable hyperparameter optimization framework.</li>
<li><a href="https://optuna.org/">Optuna</a> - Open source
hyperparameter optimization framework to automate hyperparameter
search.</li>
<li><a href="https://github.com/scikit-optimize/scikit-optimize">Scikit
Optimize</a> - Simple and efficient library to minimize expensive and
noisy black-box functions.</li>
<li><a href="https://github.com/autonomio/talos">Talos</a> -
Hyperparameter Optimization for TensorFlow, Keras and PyTorch.</li>
<li><a href="https://docs.ray.io/en/latest/tune.html">Tune</a> - Python
library for experiment execution and hyperparameter tuning at any
scale.</li>
</ul>
<h2 id="knowledge-sharing">Knowledge Sharing</h2>
<p><em>Tools for sharing knowledge to the entire team/company.</em></p>
<ul>
<li><a href="https://github.com/airbnb/knowledge-repo">Knowledge
Repo</a> - Knowledge sharing platform for data scientists and other
technical professions.</li>
<li><a href="https://kyso.io/">Kyso</a> - One place for data insights so
your entire team can learn from your data.</li>
</ul>
<h2 id="machine-learning-platform">Machine Learning Platform</h2>
<p><em>Complete machine learning platform solutions.</em></p>
<ul>
<li><a href="https://www.veritone.com/aiware/aiware-os/">aiWARE</a> -
aiWARE helps MLOps teams evaluate, deploy, integrate, scale &amp;
monitor ML models.</li>
<li><a href="https://algorithmia.com/">Algorithmia</a> - Securely govern
your machine learning operations with a healthy ML lifecycle.</li>
<li><a href="https://allegro.ai/">Allegro AI</a> - Transform ML/DL
research into products. Faster.</li>
<li><a href="https://bodywork.readthedocs.io/en/latest/">Bodywork</a> -
Deploys machine learning projects developed in Python, to
Kubernetes.</li>
<li><a href="https://cnvrg.io/">CNVRG</a> - An end-to-end machine
learning platform to build and deploy AI models at scale.</li>
<li><a href="https://dagshub.com/">DAGsHub</a> - A platform built on
open source tools for data, model and pipeline management.</li>
<li><a href="https://www.dataiku.com/">Dataiku</a> - Platform
democratizing access to data and enabling enterprises to build their own
path to AI.</li>
<li><a href="https://www.datarobot.com/">DataRobot</a> - AI platform
that democratizes data science and automates the end-to-end ML at
scale.</li>
<li><a href="https://www.dominodatalab.com/">Domino</a> - One place for
your data science tools, apps, results, models, and knowledge.</li>
<li><a href="https://edgeimpulse.com/">Edge Impulse</a> - Platform for
creating, optimizing, and deploying AI/ML algorithms for edge
devices.</li>
<li><a href="https://github.com/tensorchord/envd">envd</a> - Machine
learning development environment for data science and AI/ML engineering
teams.</li>
<li><a href="https://fedml.ai/">FedML</a> - Simplifies the workflow of
federated learning anywhere at any scale.</li>
<li><a href="https://gradient.paperspace.com/">Gradient</a> - Multicloud
CI/CD and MLOps platform for machine learning teams.</li>
<li><a href="https://www.h2o.ai/">H2O</a> - Open source leader in AI
with a mission to democratize AI for everyone.</li>
<li><a href="https://www.hopsworks.ai/">Hopsworks</a> - Open-source
platform for developing and operating machine learning models at
scale.</li>
<li><a href="https://www.iguazio.com/">Iguazio</a> - Data science
platform that automates MLOps with end-to-end machine learning
pipelines.</li>
<li><a href="https://katonic.ai/">Katonic</a> - Automate your cycle of
intelligence with Katonic MLOps Platform.</li>
<li><a href="https://www.knime.com/">Knime</a> - Create and
productionize data science using one easy and intuitive
environment.</li>
<li><a href="https://www.kubeflow.org/">Kubeflow</a> - Making
deployments of ML workflows on Kubernetes simple, portable and
scalable.</li>
<li><a href="https://lynxkite.com/">LynxKite</a> - A complete graph data
science platform for very large graphs and other datasets.</li>
<li><a href="https://github.com/ml-tooling/ml-workspace">ML
Workspace</a> - All-in-one web-based IDE specialized for machine
learning and data science.</li>
<li><a href="https://github.com/MLReef/mlreef">MLReef</a> - Open source
MLOps platform that helps you collaborate, reproduce and share your ML
work.</li>
<li><a href="https://www.modzy.com/">Modzy</a> - Deploy, connect, run,
and monitor machine learning (ML) models in the enterprise and at the
edge.</li>
<li><a href="https://neu.ro">Neu.ro</a> - MLOps platform that integrates
open-source and proprietary tools into client-oriented systems.</li>
<li><a href="https://www.omniml.ai">Omnimizer</a> - Simplifies and
accelerates MLOps by bridging the gap between ML models and edge
hardware.</li>
<li><a href="https://www.pachyderm.com/">Pachyderm</a> - Combines data
lineage with end-to-end pipelines on Kubernetes, engineered for the
enterprise.</li>
<li><a href="https://www.github.com/polyaxon/polyaxon/">Polyaxon</a> - A
platform for reproducible and scalable machine learning and deep
learning on kubernetes.</li>
<li><a href="https://aws.amazon.com/sagemaker/">Sagemaker</a> - Fully
managed service that provides the ability to build, train, and deploy ML
models quickly.</li>
<li><a href="https://www.sas.com/en_us/software/viya.html">SAS Viya</a>
- Cloud native AI, analytic and data management platform that supports
the analytics life cycle.</li>
<li><a href="https://sematic.dev">Sematic</a> - An open-source
end-to-end pipelining tool to go from laptop prototype to cloud in no
time.</li>
<li><a href="https://sigopt.com/">SigOpt</a> - A platform that makes it
easy to track runs, visualize training, and scale hyperparameter
tuning.</li>
<li><a href="https://www.truefoundry.com">TrueFoundry</a> - A
Cloud-native MLOps Platform over Kubernetes to simplify training and
serving of ML Models.</li>
<li><a href="https://valohai.com/">Valohai</a> - Takes you from POC to
production while managing the whole model lifecycle.</li>
</ul>
<h2 id="model-fairness-and-privacy">Model Fairness and Privacy</h2>
<p><em>Tools for performing model fairness and privacy in
production.</em></p>
<ul>
<li><a href="https://github.com/Trusted-AI/AIF360">AIF360</a> - A
comprehensive set of fairness metrics for datasets and machine learning
models.</li>
<li><a href="https://github.com/fairlearn/fairlearn">Fairlearn</a> - A
Python package to assess and improve fairness of machine learning
models.</li>
<li><a href="https://github.com/pytorch/opacus">Opacus</a> - A library
that enables training PyTorch models with differential privacy.</li>
<li><a href="https://github.com/tensorflow/privacy">TensorFlow
Privacy</a> - Library for training machine learning models with privacy
for training data.</li>
</ul>
<h2 id="model-interpretability">Model Interpretability</h2>
<p><em>Tools for performing model
interpretability/explainability.</em></p>
<ul>
<li><a href="https://github.com/SeldonIO/alibi">Alibi</a> - Open-source
Python library enabling ML model inspection and interpretation.</li>
<li><a href="https://github.com/pytorch/captum">Captum</a> - Model
interpretability and understanding library for PyTorch.</li>
<li><a href="https://github.com/eli5-org/eli5">ELI5</a> - Python package
which helps to debug machine learning classifiers and explain their
predictions.</li>
<li><a href="https://github.com/interpretml/interpret">InterpretML</a> -
A toolkit to help understand models and enable responsible machine
learning.</li>
<li><a href="https://github.com/marcotcr/lime">LIME</a> - Explaining the
predictions of any machine learning classifier.</li>
<li><a href="https://github.com/tensorflow/lucid">Lucid</a> - Collection
of infrastructure and tools for research in neural network
interpretability.</li>
<li><a href="https://github.com/iancovert/sage">SAGE</a> - For
calculating global feature importance using Shapley values.</li>
<li><a href="https://github.com/slundberg/shap">SHAP</a> - A game
theoretic approach to explain the output of any machine learning
model.</li>
</ul>
<h2 id="model-lifecycle">Model Lifecycle</h2>
<p><em>Tools for managing model lifecycle (tracking experiments,
parameters and metrics).</em></p>
<ul>
<li><a href="https://github.com/aimhubio/aim">Aim</a> - A super-easy way
to record, search and compare 1000s of ML training runs.</li>
<li><a href="https://github.com/Oxid15/cascade">Cascade</a> - Library of
ML-Engineering tools for rapid prototyping and experiment
management.</li>
<li><a href="https://github.com/comet-ml">Comet</a> - Track your
datasets, code changes, experimentation history, and models.</li>
<li><a href="https://guild.ai/">Guild AI</a> - Open source experiment
tracking, pipeline automation, and hyperparameter tuning.</li>
<li><a href="https://github.com/replicate/keepsake">Keepsake</a> -
Version control for machine learning with support to Amazon S3 and
Google Cloud Storage.</li>
<li><a href="https://losswise.com">Losswise</a> - Makes it easy to track
the progress of a machine learning project.</li>
<li><a href="https://mlflow.org/">Mlflow</a> - Open source platform for
the machine learning lifecycle.</li>
<li><a href="https://github.com/VertaAI/modeldb/">ModelDB</a> - Open
source ML model versioning, metadata, and experiment management.</li>
<li><a href="https://neptune.ai/">Neptune AI</a> - The most lightweight
experiment management tool that fits any workflow.</li>
<li><a href="https://github.com/IDSIA/sacred">Sacred</a> - A tool to
help you configure, organize, log and reproduce experiments.</li>
<li><a href="https://github.com/wandb/client">Weights and Biases</a> - A
tool for visualizing and tracking your machine learning
experiments.</li>
</ul>
<h2 id="model-serving">Model Serving</h2>
<p><em>Tools for serving models in production.</em></p>
<ul>
<li><a href="https://banana.dev">Banana</a> - Host your ML inference
code on serverless GPUs and integrate it into your app with one line of
code.</li>
<li><a href="https://beam.cloud">Beam</a> - Develop on serverless GPUs,
deploy highly performant APIs, and rapidly prototype ML models.</li>
<li><a href="https://github.com/bentoml/BentoML">BentoML</a> -
Open-source platform for high-performance ML model serving.</li>
<li><a href="https://github.com/ebhy/budgetml">BudgetML</a> - Deploy a
ML inference service on a budget in less than 10 lines of code.</li>
<li><a href="https://github.com/replicate/cog">Cog</a> - Open-source
tool that lets you package ML models in a standard, production-ready
container.</li>
<li><a href="https://www.cortex.dev/">Cortex</a> - Machine learning
model serving infrastructure.</li>
<li><a href="https://docs.geniusrise.ai">Geniusrise</a> - Host inference
APIs, bulk inference and fine tune text, vision, audio and multi-modal
models.</li>
<li><a href="https://github.com/gradio-app/gradio">Gradio</a> - Create
customizable UI components around your models.</li>
<li><a href="https://oracle.github.io/graphpipe">GraphPipe</a> - Machine
learning model deployment made simple.</li>
<li><a
href="https://github.com/Hydrospheredata/hydro-serving">Hydrosphere</a>
- Platform for deploying your Machine Learning to production.</li>
<li><a href="https://github.com/kubeflow/kfserving">KFServing</a> -
Kubernetes custom resource definition for serving ML models on arbitrary
frameworks.</li>
<li><a href="https://github.com/mudler/LocalAI">LocalAI</a> - Drop-in
replacement REST API that’s compatible with OpenAI API specifications
for inferencing.</li>
<li><a href="https://github.com/gojek/merlin">Merlin</a> - A platform
for deploying and serving machine learning models.</li>
<li><a href="https://github.com/iterative/mlem">MLEM</a> - Version and
deploy your ML models following GitOps principles.</li>
<li><a href="https://github.com/ml-tooling/opyrator">Opyrator</a> -
Turns your ML code into microservices with web API, interactive GUI, and
more.</li>
<li><a href="https://github.com/apache/predictionio">PredictionIO</a> -
Event collection, deployment of algorithms, evaluation, querying
predictive results via APIs.</li>
<li><a href="https://quix.io">Quix</a> - Serverless platform for
processing data streams in real-time with machine learning models.</li>
<li><a href="https://github.com/hotg-ai/rune">Rune</a> - Provides
containers to encapsulate and deploy EdgeML pipelines and
applications.</li>
<li><a href="https://www.seldon.io/">Seldon</a> - Take your ML projects
from POC to production with maximum efficiency and minimal risk.</li>
<li><a href="https://github.com/streamlit/streamlit">Streamlit</a> -
Lets you create apps for your ML projects with deceptively simple Python
scripts.</li>
<li><a href="https://www.tensorflow.org/tfx/guide/serving">TensorFlow
Serving</a> - Flexible, high-performance serving system for ML models,
designed for production.</li>
<li><a href="https://github.com/pytorch/serve">TorchServe</a> - A
flexible and easy to use tool for serving PyTorch models.</li>
<li><a href="https://github.com/triton-inference-server/server">Triton
Inference Server</a> - Provides an optimized cloud and edge inferencing
solution.</li>
<li><a href="https://github.com/vespa-engine/vespa">Vespa</a> - Store,
search, organize and make machine-learned inferences over big data at
serving time.</li>
</ul>
<h2 id="model-testing-validation">Model Testing &amp; Validation</h2>
<p><em>Tools for testing and validating models.</em></p>
<ul>
<li><a href="https://github.com/deepchecks/deepchecks">Deepchecks</a> -
Open-source package for validating ML models &amp; data, with various
checks and suites.</li>
<li><a href="https://github.com/star-whale/starwhale">Starwhale</a> - An
MLOps/LLMOps platform for model building, evaluation, and
fine-tuning.</li>
<li><a href="https://github.com/trubrics/trubrics-sdk">Trubrics</a> -
Validate machine learning with data science and domain expert
feedback.</li>
</ul>
<h2 id="optimization-tools">Optimization Tools</h2>
<p><em>Optimization tools related to model scalability in
production.</em></p>
<ul>
<li><a href="https://github.com/huggingface/accelerate">Accelerate</a> -
A simple way to train and use PyTorch models with multi-GPU, TPU,
mixed-precision.</li>
<li><a href="https://dask.org/">Dask</a> - Provides advanced parallelism
for analytics, enabling performance at scale for the tools you
love.</li>
<li><a href="https://github.com/microsoft/DeepSpeed">DeepSpeed</a> -
Deep learning optimization library that makes distributed training easy,
efficient, and effective.</li>
<li><a href="https://uber.github.io/fiber/">Fiber</a> - Python
distributed computing library for modern computer clusters.</li>
<li><a href="https://github.com/horovod/horovod">Horovod</a> -
Distributed deep learning training framework for TensorFlow, Keras,
PyTorch, and Apache MXNet.</li>
<li><a href="https://mahout.apache.org/">Mahout</a> - Distributed linear
algebra framework and mathematically expressive Scala DSL.</li>
<li><a href="https://spark.apache.org/mllib/">MLlib</a> - Apache Spark’s
scalable machine learning library.</li>
<li><a href="https://github.com/modin-project/modin">Modin</a> - Speed
up your Pandas workflows by changing a single line of code.</li>
<li><a href="https://github.com/nebuly-ai/nebullvm">Nebullvm</a> -
Easy-to-use library to boost AI inference.</li>
<li><a href="https://github.com/nebuly-ai/nos">Nos</a> - Open-source
module for running AI workloads on Kubernetes in an optimized way.</li>
<li><a href="https://github.com/uber/petastorm">Petastorm</a> - Enables
single machine or distributed training and evaluation of deep learning
models.</li>
<li><a href="https://rapids.ai/index.html">Rapids</a> - Gives the
ability to execute end-to-end data science and analytics pipelines
entirely on GPUs.</li>
<li><a href="https://github.com/ray-project/ray">Ray</a> - Fast and
simple framework for building and running distributed applications.</li>
<li><a href="http://singa.apache.org/en/index.html">Singa</a> - Apache
top level project, focusing on distributed training of DL and ML
models.</li>
<li><a href="https://github.com/EpistasisLab/tpot">Tpot</a> - Automated
ML tool that optimizes machine learning pipelines using genetic
programming.</li>
</ul>
<h2 id="simplification-tools">Simplification Tools</h2>
<p><em>Tools related to machine learning simplification and
standardization.</em></p>
<ul>
<li><a href="https://chassisml.io">Chassis</a> - Turns models into
ML-friendly containers that run just about anywhere.</li>
<li><a href="https://github.com/a3data/hermione">Hermione</a> - Help
Data Scientists on setting up more organized codes, in a quicker and
simpler way.</li>
<li><a href="https://github.com/facebookresearch/hydra">Hydra</a> - A
framework for elegantly configuring complex applications.</li>
<li><a href="https://github.com/databricks/koalas">Koalas</a> - Pandas
API on Apache Spark. Makes data scientists more productive when
interacting with big data.</li>
<li><a href="https://github.com/uber/ludwig">Ludwig</a> - Allows users
to train and test deep learning models without the need to write
code.</li>
<li><a href="https://github.com/aporia-ai/mlnotify">MLNotify</a> - No
need to keep checking your training, just one import line and you’ll
know the second it’s done.</li>
<li><a href="https://pycaret.org/">PyCaret</a> - Open source, low-code
machine learning library in Python.</li>
<li><a href="https://github.com/Kenza-AI/sagify">Sagify</a> - A CLI
utility to train and deploy ML/DL models on AWS SageMaker.</li>
<li><a href="https://github.com/ploomber/soopervisor">Soopervisor</a> -
Export ML projects to Kubernetes (Argo workflows), Airflow, AWS Batch,
and SLURM.</li>
<li><a href="https://github.com/ploomber/soorgeon">Soorgeon</a> -
Convert monolithic Jupyter notebooks into maintainable pipelines.</li>
<li><a
href="https://github.com/jrieke/traingenerator">TrainGenerator</a> - A
web app to generate template code for machine learning.</li>
<li><a href="https://github.com/apple/turicreate">Turi Create</a> -
Simplifies the development of custom machine learning models.</li>
</ul>
<h2 id="visual-analysis-and-debugging">Visual Analysis and
Debugging</h2>
<p><em>Tools for performing visual analysis and debugging of ML/DL
models.</em></p>
<ul>
<li><a href="https://www.aporia.com/">Aporia</a> - Observability with
customized monitoring and explainability for ML models.</li>
<li><a href="https://www.arize.com/">Arize</a> - A free end-to-end ML
observability and model monitoring platform.</li>
<li><a href="https://github.com/comet-ml/comet-llm">CometLLM</a> -
Track, visualize, and evaluate your LLM prompts and chains in one
easy-to-use UI.</li>
<li><a href="https://github.com/evidentlyai/evidently">Evidently</a> -
Interactive reports to analyze ML models during validation or production
monitoring.</li>
<li><a href="https://www.fiddler.ai/">Fiddler</a> - Monitor, explain,
and analyze your AI in production.</li>
<li><a href="https://github.com/uber/manifold">Manifold</a> - A
model-agnostic visual debugging tool for machine learning.</li>
<li><a href="https://github.com/NannyML/nannyml">NannyML</a> - Algorithm
capable of fully capturing the impact of data drift on performance.</li>
<li><a href="https://github.com/lutzroeder/netron">Netron</a> -
Visualizer for neural network, deep learning, and machine learning
models.</li>
<li><a href="https://phoenix.arize.com">Phoenix</a> - MLOps in a
Notebook for troubleshooting and fine-tuning generative LLM, CV, and
tabular models.</li>
<li><a href="https://www.superwise.ai">Superwise</a> - Fully automated,
enterprise-grade model observability in a self-service SaaS
platform.</li>
<li><a href="https://github.com/whylabs/whylogs">Whylogs</a> - The open
source standard for data logging. Enables ML monitoring and
observability.</li>
<li><a
href="https://github.com/DistrictDataLabs/yellowbrick">Yellowbrick</a> -
Visual analysis and diagnostic tools to facilitate machine learning
model selection.</li>
</ul>
<h2 id="workflow-tools">Workflow Tools</h2>
<p><em>Tools and frameworks to create workflows or pipelines in the
machine learning context.</em></p>
<ul>
<li><a href="https://github.com/argoproj/argo">Argo</a> - Open source
container-native workflow engine for orchestrating parallel jobs on
Kubernetes.</li>
<li><a
href="https://www.veritone.com/applications/automate-studio/">Automate
Studio</a> - Rapidly build &amp; deploy AI-powered workflows.</li>
<li><a href="https://github.com/couler-proj/couler">Couler</a> - Unified
interface for constructing and managing workflows on different workflow
engines.</li>
<li><a href="https://github.com/dstackai/dstack">dstack</a> - An
open-core tool to automate data and training workflows.</li>
<li><a href="https://flyte.org/">Flyte</a> - Easy to create concurrent,
scalable, and maintainable workflows for machine learning.</li>
<li><a href="https://github.com/dagworks-inc/hamilton">Hamilton</a> - A
scalable general purpose micro-framework for defining dataflows.</li>
<li><a href="https://github.com/kubeflow-kale/kale">Kale</a> - Aims at
simplifying the Data Science experience of deploying Kubeflow Pipelines
workflows.</li>
<li><a href="https://github.com/quantumblacklabs/kedro">Kedro</a> -
Library that implements software engineering best-practice for data and
ML pipelines.</li>
<li><a href="https://github.com/spotify/luigi">Luigi</a> - Python module
that helps you build complex pipelines of batch jobs.</li>
<li><a href="https://metaflow.org/">Metaflow</a> - Human-friendly lib
that helps scientists and engineers build and manage data science
projects.</li>
<li><a href="https://github.com/mlrun/mlrun">MLRun</a> - Generic
mechanism for data scientists to build, run, and monitor ML tasks and
pipelines.</li>
<li><a href="https://github.com/orchest/orchest/">Orchest</a> - Visual
pipeline editor and workflow orchestrator with an easy to use UI and
based on Kubernetes.</li>
<li><a href="https://github.com/ploomber/ploomber">Ploomber</a> - Write
maintainable, production-ready pipelines. Develop locally, deploy to the
cloud.</li>
<li><a href="https://docs.prefect.io/">Prefect</a> - A workflow
management system, designed for modern infrastructure.</li>
<li><a href="https://github.com/instill-ai/vdp">VDP</a> - An open-source
tool to seamlessly integrate AI for unstructured data into the modern
data stack.</li>
<li><a href="https://github.com/maiot-io/zenml">ZenML</a> - An
extensible open-source MLOps framework to create reproducible
pipelines.</li>
</ul>
<hr />
<h1 id="resources">Resources</h1>
<p>Where to discover new tools and discuss about existing ones.</p>
<h2 id="articles">Articles</h2>
<ul>
<li><a
href="https://databaseline.tech/a-tour-of-end-to-end-ml-platforms/">A
Tour of End-to-End Machine Learning Platforms</a> (Databaseline)</li>
<li><a href="https://martinfowler.com/articles/cd4ml.html">Continuous
Delivery for Machine Learning</a> (Martin Fowler)</li>
<li><a
href="https://azure.microsoft.com/mediahandler/files/resourcefiles/gigaom-Delivering-on-the-Vision-of-MLOps/Delivering%20on%20the%20Vision%20of%20MLOps.pdf">Delivering
on the Vision of MLOps: A maturity-based approach</a> (GigaOm)</li>
<li><a href="https://arxiv.org/abs/2205.02302">Machine Learning
Operations (MLOps): Overview, Definition, and Architecture</a>
(arXiv)</li>
<li><a
href="https://cloud.google.com/solutions/machine-learning/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning">MLOps:
Continuous delivery and automation pipelines in machine learning</a>
(Google)</li>
<li><a
href="https://towardsdatascience.com/ml-ops-machine-learning-as-an-engineering-discipline-b86ca4874a3f">MLOps:
Machine Learning as an Engineering Discipline</a> (Medium)</li>
<li><a
href="https://developers.google.com/machine-learning/guides/rules-of-ml">Rules
of Machine Learning: Best Practices for ML Engineering</a> (Google)</li>
<li><a
href="https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/aad9f93b86b7addfea4c419b9100c6cdd26cacea.pdf">The
ML Test Score: A Rubric for ML Production Readiness and Technical Debt
Reduction</a> (Google)</li>
<li><a
href="https://blogs.nvidia.com/blog/2020/09/03/what-is-mlops/">What Is
MLOps?</a> (NVIDIA)</li>
</ul>
<h2 id="books">Books</h2>
<ul>
<li><a
href="https://www.amazon.com/Beginning-MLOps-MLFlow-SageMaker-Microsoft/dp/1484265483">Beginning
MLOps with MLFlow</a> (Apress)</li>
<li><a
href="https://www.oreilly.com/library/view/building-machine-learning/9781492053187">Building
Machine Learning Pipelines</a> (O’Reilly)</li>
<li><a
href="https://www.oreilly.com/library/view/building-machine-learning/9781492045106">Building
Machine Learning Powered Applications</a> (O’Reilly)</li>
<li><a href="https://www.amazon.com/gp/product/6180033773">Deep Learning
in Production</a> (AI Summer)</li>
<li><a
href="https://www.oreilly.com/library/view/designing-machine-learning/9781098107956">Designing
Machine Learning Systems</a> (O’Reilly)</li>
<li><a
href="https://www.packtpub.com/product/engineering-mlops/9781800562882">Engineering
MLOps</a> (Packt)</li>
<li><a
href="https://www.oreilly.com/library/view/implementing-mlops-in/9781098136574">Implementing
MLOps in the Enterprise</a> (O’Reilly)</li>
<li><a
href="https://www.oreilly.com/library/view/introducing-mlops/9781492083283">Introducing
MLOps</a> (O’Reilly)</li>
<li><a
href="https://www.oreilly.com/library/view/kubeflow-for-machine/9781492050117">Kubeflow
for Machine Learning</a> (O’Reilly)</li>
<li><a
href="https://www.oreilly.com/library/view/kubeflow-operations-guide/9781492053262">Kubeflow
Operations Guide</a> (O’Reilly)</li>
<li><a
href="https://www.oreilly.com/library/view/machine-learning-design/9781098115777">Machine
Learning Design Patterns</a> (O’Reilly)</li>
<li><a
href="https://www.manning.com/books/machine-learning-engineering-in-action">Machine
Learning Engineering in Action</a> (Manning)</li>
<li><a
href="https://www.oreilly.com/library/view/ml-ops-operationalizing/9781492074663">ML
Ops: Operationalizing Data Science</a> (O’Reilly)</li>
<li><a
href="https://www.manning.com/books/mlops-engineering-at-scale">MLOps
Engineering at Scale</a> (Manning)</li>
<li><a
href="https://link.springer.com/book/10.1007/978-1-4842-9642-4">MLOps
Lifecycle Toolkit</a> (Apress)</li>
<li><a
href="https://www.packtpub.com/product/practical-deep-learning-at-scale-with-mlflow/9781803241333">Practical
Deep Learning at Scale with MLflow</a> (Packt)</li>
<li><a
href="https://www.oreilly.com/library/view/practical-mlops/9781098103002">Practical
MLOps</a> (O’Reilly)</li>
<li><a
href="https://www.packtpub.com/product/production-ready-applied-deep-learning/9781803243665">Production-Ready
Applied Deep Learning</a> (Packt)</li>
<li><a
href="https://www.oreilly.com/library/view/reliable-machine-learning/9781098106218">Reliable
Machine Learning</a> (O’Reilly)</li>
<li><a
href="https://www.packtpub.com/product/the-machine-learning-solutions-architect-handbook/9781801072168">The
Machine Learning Solutions Architect Handbook</a> (Packt)</li>
</ul>
<h2 id="events">Events</h2>
<ul>
<li><a href="https://www.applyconf.com/">apply() - The ML data
engineering conference</a></li>
<li><a
href="https://www.youtube.com/playlist?list=PLH8M0UOY0uy6d_n3vEQe6J_gRBUrISF9m">MLOps
Conference - Keynotes and Panels</a></li>
<li><a href="https://mlopsworld.com/">MLOps World: Machine Learning in
Production Conference</a></li>
<li><a href="https://normconf.com/">NormConf - The Normcore Tech
Conference</a></li>
<li><a href="https://mlsys.stanford.edu/">Stanford MLSys Seminar
Series</a></li>
</ul>
<h2 id="other-lists">Other Lists</h2>
<ul>
<li><a href="https://github.com/eugeneyan/applied-ml">Applied
ML</a></li>
<li><a
href="https://github.com/hibayesian/awesome-automl-papers">Awesome
AutoML Papers</a></li>
<li><a href="https://github.com/windmaple/awesome-AutoML">Awesome
AutoML</a></li>
<li><a href="https://github.com/academic/awesome-datascience">Awesome
Data Science</a></li>
<li><a href="https://github.com/kelvins/awesome-dataops">Awesome
DataOps</a></li>
<li><a
href="https://github.com/ChristosChristofidis/awesome-deep-learning">Awesome
Deep Learning</a></li>
<li><a href="https://github.com/leomaurodesenv/game-datasets">Awesome
Game Datasets</a> (includes AI content)</li>
<li><a
href="https://github.com/josephmisiti/awesome-machine-learning">Awesome
Machine Learning</a></li>
<li><a href="https://github.com/visenger/awesome-mlops">Awesome
MLOps</a></li>
<li><a
href="https://github.com/EthicalML/awesome-production-machine-learning">Awesome
Production Machine Learning</a></li>
<li><a href="https://github.com/vinta/awesome-python">Awesome
Python</a></li>
<li><a
href="https://github.com/ahkarami/Deep-Learning-in-Production">Deep
Learning in Production</a></li>
</ul>
<h2 id="podcasts">Podcasts</h2>
<ul>
<li><a href="https://how-ai-built-this.captivate.fm/">How AI Built
This</a></li>
<li><a href="https://kubernetespodcast.com/">Kubernetes Podcast from
Google</a></li>
<li><a
href="https://podcasts.google.com/?feed=aHR0cHM6Ly9zb2Z0d2FyZWVuZ2luZWVyaW5nZGFpbHkuY29tL2NhdGVnb3J5L21hY2hpbmUtbGVhcm5pbmcvZmVlZC8">Machine
Learning – Software Engineering Daily</a></li>
<li><a
href="https://podcasts.google.com/?feed=aHR0cHM6Ly9hbmNob3IuZm0vcy8xNzRjYjFiOC9wb2RjYXN0L3Jzcw">MLOps.community</a></li>
<li><a href="https://podcast.zenml.io/">Pipeline Conversation</a></li>
<li><a href="https://changelog.com/practicalai">Practical AI: Machine
Learning, Data Science</a></li>
<li><a href="https://twimlai.com/">This Week in Machine Learning &amp;
AI</a></li>
<li><a
href="https://www.youtube.com/playlist?list=PL4-eEhdXDO5F9Myvh41EeUh7oCgzqFRGk">True
ML Talks</a></li>
</ul>
<h2 id="slack">Slack</h2>
<ul>
<li><a href="https://kubeflow.slack.com/#/">Kubeflow Workspace</a></li>
<li><a href="https://mlops-community.slack.com">MLOps Community
Wokspace</a></li>
</ul>
<h2 id="websites">Websites</h2>
<ul>
<li><a href="http://featurestore.org/">Feature Stores for ML</a></li>
<li><a href="https://github.com/GokuMohandas/Made-With-ML">Made with
ML</a></li>
<li><a href="https://ml-ops.org/">ML-Ops</a></li>
<li><a href="https://mlops.community/">MLOps Community</a></li>
<li><a href="https://mlops-guide.github.io/">MLOps Guide</a></li>
<li><a href="https://mlopsnow.com">MLOps Now</a></li>
</ul>
<h1 id="contributing">Contributing</h1>
<p>All contributions are welcome! Please take a look at the <a
href="https://github.com/kelvins/awesome-mlops/blob/main/CONTRIBUTING.md">contribution
guidelines</a> first.</p>