Awesome

⭐️ Why Agenta?

Agenta is an end-to-end LLM developer platform. It provides the tools for prompt engineering and management, ⚖️ evaluation, human annotation, and :rocket: deployment. All without imposing any restrictions on your choice of framework, library, or model.

Agenta allows developers and product teams to collaborate in building production-grade LLM-powered applications in less time.

With Agenta, you can:

• 🧪 Experiment and compare prompts on any LLM workflow (chain-of-prompts, Retrieval Augmented Generation (RAG), LLM agents...)
• ✍️ Collect and annotate golden test sets for evaluation
• 📈 Evaluate your application with pre-existing or custom evaluators
• 🔍 Annotate and A/B test your applications with human feedback
• 🤝 Collaborate with product teams for prompt engineering and evaluation
• 🚀 Deploy your application in one-click in the UI, through CLI, or through github workflows.

Works with any LLM app workflow

Agenta enables prompt engineering and evaluation on any LLM app architecture:

• Chain of prompts
• RAG
• Agents

It works with any framework such as Langchain, LlamaIndex and any LLM provider (openAI, Cohere, Mistral).

Quick Start

Get started for free

Explore the Docs

Create your first application in one-minute

Create an application using Langchain

Self-host agenta

Check the Cookbook

Features

Playground	Evaluation
Compare and version prompts for any LLM app, from single prompt to agents. <br/> <video src="https://github.com/Agenta-AI/agenta/assets/4510758/8b736d2b-7c61-414c-b534-d95efc69134c" controls="controls" style="max-width:100%;">	Define test sets, then evaluate manually or programmatically your different variants.<br/> <video src="https://github.com/Agenta-AI/agenta/assets/4510758/8c6997c6-da87-46ad-a81f-e15e277263d2" controls="controls" style="max-width:100%;">
Human annotation	Deployment
Use Human annotator to A/B test and score your LLM apps. <br/> <img width="750" alt="Screenshot 2024-01-28 at 12 57 46" src="https://github.com/Agenta-AI/agenta/assets/4510758/bf62a697-bf19-4ba9-850e-742fbfb75424">	When you are ready, deploy your LLM applications as APIs in one click.<br/>

Enterprise Support

Contact us here for enterprise support and early access to agenta self-managed enterprise with Kubernetes support. <br/><br/> <a href="https://cal.com/mahmoud-mabrouk-ogzgey/demo"><img src="https://cal.com/book-with-cal-dark.svg" alt="Book us"></a>

Disabling Anonymized Tracking

By default, Agenta automatically reports anonymized basic usage statistics. This helps us understand how Agenta is used and track its overall usage and growth. This data does not include any sensitive information.

To disable anonymized telemetry, follow these steps:

• For web: Set TELEMETRY_TRACKING_ENABLED to false in your agenta-web/.env file.
• For CLI: Set telemetry_tracking_enabled to false in your ~/.agenta/config.toml file.

After making this change, restart Agenta Compose.

⭐️ Join Our Team

• Founding Product Engineer Frontend

Contributing

We warmly welcome contributions to Agenta. Feel free to submit issues, fork the repository, and send pull requests.

We are usually hanging in our Slack. Feel free to join our Slack and ask us anything

Check out our Contributing Guide for more information.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

This project follows the all-contributors specification. Contributions of any kind are welcome!

Attribution: Testing icons created by Freepik - Flaticon