Awesome

modelstore

modelstore is a Python library that allows you to version, export, save and download machine learning models in your choice of storage.

💭 Give us feedback by completing this survey: https://forms.gle/XShU3zrZcnLRWsk36

modelstore is an open source model registry

✅ No tracking server required

Store models on a local file system or in a bucket
Support for multiple clouds (AWS, GCP, Azure)

✅ Upload and version all your models

Models are versioned on each upload
Replaces all the boiler plate code you need to save models

✅ Manage models by domains and states

List models in a domain
Create model states and manage which state a model is in

✅ Download or load straight into memory

Download models by id
Load models straight from your storage back into memory

✅ Use as a command line tool

Download models from the command line

For more details, please refer to the documentation.

modelstore is being built in the open

💬 Come and find us in the MLOps Community Slack's #oss-modelstore channel.

Installation

pip install modelstore

Supported storage types

AWS S3 Bucket (example)
Azure Blob Storage (example)
Google Cloud Storage Bucket (example)
Any s3-compatible object storage that you can access via MinIO
A filesystem directory (example)

Supported machine learning libraries

Annoy
Catboost
CausalML
Fast.AI
Gensim
Keras
LightGBM
Mxnet
Onnx
Prophet
PyTorch
PyTorch Lightning
Scikit Learn
Skorch
Shap
Spark ML Lib
Tensorflow
Transformers - there are several examples in this directory
XGBoost

Is there a machine learning framework that is missing?

Save your model and then upload it as a raw file.
Feel free to open an issue

Example Usage

Colab Notebook

There is a full example in this Colab notebook.

Python Script

from modelstore import ModelStore
# And your other imports

# Train your model
clf = RandomForestClassifier(n_estimators=10)
clf = clf.fit(X, Y)

# Create a model store that uses a one of the storage options
# In this example, the model store is created with a GCP bucket
model_store = ModelStore.from_gcloud(
   project_name="my-project",
   bucket_name="my-bucket",
)

# Upload the archive to your model store
domain = "example-model"
meta_data = model_store.upload(domain, model=clf)

# Print the meta-data about the model
print(json.dumps(meta_data, indent=4))

# Load the model back!
clf = model_store.load(domain=model_domain, model_id=meta["model"]["model_id"])

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.