Home

Awesome

Ianvs

CI LICENSE SCAN LICENSE

Ianvs is a distributed synergy AI benchmarking project incubated in KubeEdge SIG AI. Ianvs aims to test the performance of distributed synergy AI solutions following recognized standards, in order to facilitate more efficient and effective development. More detailedly, Ianvs prepares not only test cases with datasets and corresponding algorithms, but also benchmarking tools including simulation and hyper-parameter searching. Ianvs also reveals best practices for developers and end users with presentation tools including leaderboards and test reports.

Scope

The distributed synergy AI benchmarking Ianvs aims to test the performance of distributed synergy AI solutions following recognized standards, in order to facilitate more efficient and effective development.

The scope of Ianvs includes

Architecture

The architectures and related concepts are shown in the below figure. The ianvs is designed to run within a single node. Critical components include

More details on Ianvs components:

  1. Test-Environment Manager supports the CRUD of Test environments, which basically includes
    • Algorithm-wise configuration
      • Public datasets
      • Pre-processing algorithms
      • Feature engineering algorithms
      • Post-processing algorithms like metric computation
    • System-wise configuration
      • Overall architecture
      • System constraints or budgets
        • End-to-end cross-node
        • Per node
  2. Test-case Controller, which includes but is not limited to the following components
    • Templates of common distributed-synergy-AI paradigms, which can help the developer to prepare their test case without too much effort. Such paradigms include edge-cloud synergy joint inference, incremental learning, federated learning, and lifelong learning.
    • Simulation tools. Develop simulated test environments for test cases
    • Other tools to assist test-case generation. For instance, prepare test cases based on a given range of hyper-parameters.
  3. Story Manager, which includes but is not limited to the following components
    • Leaderboard generation
    • Test report generation

Guides

Documents

Documentation is located on readthedoc.io. The documents include the quick start, guides, dataset descriptions, algorithms, user interfaces, stories, and roadmap.

Installation

Follow the Ianvs installation document to install Ianvs.

Examples

Roadmap

Meeting

Routine Community Meeting for KubeEdge SIG AI runs weekly:

Resources:

Contact

<!-- If you need support, start with the [troubleshooting guide](./docs/troubleshooting.md), and work your way through the process that we've outlined. -->

If you have questions, feel free to reach out to us in the following ways:

Contributing

If you're interested in being a contributor and want to get involved in developing the Ianvs code, please see CONTRIBUTING for details on submitting patches and the contribution workflow.

License

Ianvs is under the Apache 2.0 license. See the LICENSE file for details.