AI Model Stores

What’s a model store?
The model store is a central storage for data scientists to take and manage their models and experiments, including the model files, artifacts, and metadata.

With model stores, you control the complexity of managing multiple machine learning models. Including below:
1. Compare multiple, newly trained model versions against existing deployed versions;
2. Compare completely new models against versions of other models on labeled data;
3. Track model performance over time;
4. Track organization-wide experiments;
5. Manage serving needs for organization-wide machine learning models.

Why do you need a model store for your MLOps projects?
1. Reproducibility of the model(s).
2. Ensuring the model(s) is production-ready.
3. Managing the model(s) effectively.

Model stores guarantee reproducibility in the following ways:
  Tracking and collecting experiment– and ML pipeline-related metadata (experiment author/owner, description, etc.).
  Collecting dataset metadata, including version, location, and description of the dataset. Also, how a user chose the data or where the data links to in the feature store.
  Collecting model artifacts, metadata (packages, frameworks, language, environment files, git files, etc), and configuration files.
  Collecting container artifacts.
  Project documentation, including demos and examples on how to run a model.

What you can find in a model store?
1. Diverse metadata: From models, data, and experiments.
Artifacts: Like the metadata, the store contains all artifacts relevant to how you develop, deploy, and manage models.
2. Documentation and reporting tools: Documentation is crucial for reviews and reproducible projects. Model stores enable documentation relevant to how you develop your model, deploy, and manage them.
3. Catalog: The information in the model stores needs to be searchable, and the catalog enables this. Searching for models to use? How about related metadata? Searching for which models were trained on a particular dataset? The catalog makes the store searchable.
4. Staging tools: Another feature of the model store is the staging integration tests it can carry out on models. You can find tools for staging models for testing within the model store.
5. Automation tools: One of the goals of model stores is to automate some repetitive tasks after you have trained and validated a model to increase the productivity of teams deploying lots of models. Within the store you can find automation tools and workflows that enable this process.

Different Model Stores
1. Modelstore
Modelstore (how original, Neal! ) is an open-source Python library that allows you to version, export, and save/retrieve machine learning models to and from your filesystem or a cloud storage provider (AWS or GCP).

2. ClearML Model Stores
ClearML states on their website that it’s the only open-source tool to manage all your MLOps in a unified and robust platform providing collaborative experiment management, powerful orchestration, easy-to-build data stores, and one-click model deployment.

3. MLflow Model Registry
MLflow is an open-source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. The MLflow Model Registry component is a centralized model store, set of APIs, and UI, to collaboratively manage the entire lifecycle of an MLflow Model across data teams.
Some of the features of the MLflow Model Registry include:
Provides a central repository to store and manage uniquely named registered models for collaboration and visibility across data teams.
Provides a UI and API for registry operations and a smooth workflow experience.
Allows multiple versions of the model in different stages environments (staging and production environments).
Allows transition and model promotion schemes across different environments and stages. Models can be moved from staging, loaded to the production environment, rolled back, and retired or archived.
Integrated with CI/CD pipelines to quickly load a specific model version for testing, review, approval, release, and rollback.
Model lineage tracking feature that provides model description, lineage, and activity.

4. neptune.ai
Neptune is a metadata store for MLOps, built for research and production teams that run many experiments. It gives you a central place to log, store, display, organize, compare, and query all metadata generated during the machine learning lifecycle. Neptune is more of a metadata store than an actual artifact store

5. Verta.ai
Verta.ai uses a suite of tools to empower data science and machine learning teams to rapidly develop and deploy production-ready models, thereby enabling efficient integration of ML into various products. One of the tools in their platform is the Model Registry, a central repository to find, publish, collaborate on and use production-ready models.