Awesome

Aeromancy

Aeromancy is an opinionated philosophy and open-sourced framework that closely tracks experimental runtime environments for more reproducible machine learning. In existing experiment trackers, it’s easy to miss important details about how an experiment was run, e.g., which version of a dataset was used as input or the exact versions of library dependencies. Missing these details can make replicability more difficult. Aeromancy aims to make this process smoother by providing both new infrastructure (a more comprehensive versioning scheme including both system runtimes and external datasets) and a corresponding set of best practices to ensure experiments are maximally trackable.

In its current form, Aeromancy requires a fairly specific software stack: (hey, we said it was opinionated)

• Experiment tracker: Weights and Biases
• Object storage (artifacts): S3-compatible, e.g., Ceph
• Virtualization: Docker
• Python Package Manager: pdm
• Revision Control: Git

Aeromancy at SciPy 2024

Check out our abstract and poster:

Documentation

• If you're new to Aeromancy, start here!
• In the Developer Reference section of the documentation, we include some design docs which provide an architectural overview and a glossary of terms.
• To see autogenerated docs for code from this repo, you'll need to start a local doc server (pdm doc).
• Want to get involved? We have starting points in our Contributor Guidelines.

Note: Aeromancy documentation is in a very early state. As this is a pre-release support may be limited.

Common development commands

• pdm lint: Run pre-commit linters
• pdm test: Run test suite
• pdm doc: Start doc server (see also the public version for the latest release)