Awesome

<div align="left"> <h1>Mage AI</h1> <p>Give your data team magical powers.</p>

</div> <br /> <a href="https://mage.ai/pro"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://github.com/mage-ai/assets/blob/main/oss/hero.png?raw=true"> <img alt="Mage AI hero" src="https://github.com/mage-ai/assets/blob/main/oss/hero.png?raw=true" /> </picture> </a> <br /> <br />

Mage is a hybrid framework for transforming and integrating data. It combines the best of both worlds: the flexibility of notebooks with the rigor of modular code.

<br />

Extract and synchronize data from 3rd party sources.
Transform data with real-time and batch pipelines using Python, SQL, and R.
Load data into your data warehouse or data lake using our pre-built connectors.
Run, monitor, and orchestrate thousands of pipelines without losing sleep.

<br />

Plus hundreds of enterprise-class features, infrastructure innovations, and magical surprises.

Available in two spellbinding versions

<br /> <table style="width: 100%; border: none;" cellspacing="0" cellpadding="0" border="0"> <tr> <td rowspan="2"> <a href="https://mage.ai/pro"> <img alt="Mage Pro" width="124" align="left" src="https://github.com/mage-ai/assets/blob/main/oss/card-pro.png?raw=true" /> </a> <b>For teams.</b> Fully managed platform for integrating and transforming data. </td> <td rowspan="2"> <a href="https://github.com/mage-ai/mage-ai?tab=readme-ov-file#its-magic"> <img alt="Mage OSS" width="124" align="left" src="https://github.com/mage-ai/assets/blob/main/oss/card-oss.png?raw=true" /> </a> <b>Self-hosted.</b> System to build, run, and manage data pipelines. </td> </tr> </table> <br /> <a href="https://mage.ai/pro"> <img alt="Try out Mage Pro" src="https://img.shields.io/badge/try%20it%20out%20now-black.svg?style=for-the-badge&logo=artifacthub&labelColor=6A35FF&logoColor=FFFFFF&label=mage%20pro&color=6A35FF"> </a> <br /> <br /> <div align="left">

It’s magic.

<p align="left"> For documentation on getting started, how to develop, and how to deploy to production check out the live <br /> <a href="https://docs.mage.ai?source=github" target="_blank" ><b>Developer documentation portal</b></a>. </p> <br />

🏃‍♀️ Install

The recommended way to install the latest version of Mage is through Docker with the following command:

docker pull mageai/mageai:latest

You can also install Mage using pip or conda, though this may cause dependency issues without the proper environment.

pip install mage-ai

conda install -c conda-forge mage-ai

Looking for help? The fastest way to get started is by checking out our documentation here.

Looking for quick examples? Open a demo project right in your browser or check out our guides.

🎮 Demo

Live demo

Build and run a data pipeline with our <b>demo app</b>.

WARNING

The live demo is public to everyone, please don’t save anything sensitive (e.g. passwords, secrets, etc).

Demo video (5 min)

<sub><i>Click the image to play video</i></sub>

<br />

🔮 Features


🎶	<b>Orchestration</b>	Schedule and manage data pipelines with observability.
📓	<b>Notebook</b>	Interactive Python, SQL, & R editor for coding data pipelines.
🏗️	<b>Data integrations</b>	Synchronize data from 3rd party sources to your internal destinations.
🚰	<b>Streaming pipelines</b>	Ingest and transform real-time data.
❎	<b>dbt</b>	Build, run, and manage your dbt models with Mage.

<br />

<b>A sample data pipeline defined across 3 files ➝</b>

<br />

Load data ➝

@data_loader
def load_csv_from_file() -> pl.DataFrame:
    return pl.read_csv('default_repo/titanic.csv')

Transform data ➝

@transformer
def select_columns_from_df(df: pl.DataFrame, *args) -> pl.DataFrame:
    return df[['Age', 'Fare', 'Survived']]

Export data ➝

@data_exporter
def export_titanic_data_to_disk(df: pl.DataFrame) -> None:
    df.to_csv('default_repo/titanic_transformed.csv')

<br />