Home

Awesome

<p align=center> <img height="150px" src="docs/static/img/getsynth_favicon.png"/> </p> <p align=center> <b>The Declarative Data Generator</b> </p> <br> <p align=center> <a href="https://getsynth.com/docs"><img alt="docs" src="https://img.shields.io/badge/doc-reference-orange"></a> <a href="https://github.com/getsynth/synth/blob/master/LICENSE"><img alt="license" src="https://img.shields.io/badge/license-Apache_2.0-green.svg"></a> <a href="https://github.com/getsynth/synth/search?l=rust"><img alt="language" src="https://img.shields.io/badge/language-Rust-orange.svg"></a> <a href="https://github.com/getsynth/synth/actions"><img alt="build status" src="https://img.shields.io/github/workflow/status/getsynth/synth/synth%20public%20cachix"/></a> <a href="https://discord.gg/SBb2wGgqKR"> <img alt="discord" src="https://img.shields.io/discord/1005909761235107961?logo=discord"/></a> <img src="https://img.shields.io/github/all-contributors/getsynth/synth" alt="Synth open source contributors"/> </p>

Synth is a tool for generating realistic data using a declarative data model. Synth is database agnostic and can scale to millions of rows of data.

Why Synth

Synth answers a simple question. There are so many ways to consume data, why are there no frameworks for generating data?

Synth provides a robust, declarative framework for specifying constraint based data generation, solving the following problems developers face on the regular:

  1. You're creating an App from scratch and have no way to populate your fresh schema with correct, realistic data.
  2. You're doing integration testing / QA on production data, but you know it is bad practice, and you really should not be doing that.
  3. You want to see how your system will scale if your database suddenly has 10x the amount of data.

Synth solves exactly these problems with a flexible declarative data model which you can version control in git, peer review, and automate.

Key Features

The key features of Synth are:

Status

We are currently in Public Alpha. Watch "releases" of this repo to get notified of major updates.

Installation & Getting Started

On Linux and MacOS you can get started with the one-liner:

# Optional, set install path
$ export SYNTH_INSTALL_PATH=~/bin
$ curl -sSL https://getsynth.com/install | sh

For more installation options, check out the docs.

Examples

Building a data model from scratch

To start generating data without having a source to import from, you need to add Synth schema files to a namespace directory:

To get started we'll create a namespace directory for our data model and call it my_app:

$ mkdir my_app

Next let's create a users collection using Synth's configuration language, and put it into my_app/users.json:

{
    "type": "array",
    "length": {
        "type": "number",
        "constant": 1
    },
    "content": {
        "type": "object",
        "id": {
            "type": "number",
            "id": {}
        },
        "email": {
            "type": "string",
            "faker": {
                "generator": "safe_email"
            }
        },
        "joined_on": {
            "type": "date_time",
            "format": "%Y-%m-%d",
            "subtype": "naive_date",
            "begin": "2010-01-01",
            "end": "2020-01-01"
        }
    }
}

Finally, generate data using the synth generate command:

$ synth generate my_app/ --size 2 | jq
{
  "users": [
    {
      "email": "patricia40@jordan.com",
      "id": 1,
      "joined_on": "2014-12-14"
    },
    {
      "email": "benjamin00@yahoo.com",
      "id": 2,
      "joined_on": "2013-04-06"
    }
  ]
}

Building a data model from an external database

If you have an existing database, Synth can automatically generate a data model by inspecting the database.

You can use the synth import command to automatically generate Synth schema files from your Postgres, MySQL or MongoDB database:

$ synth import tpch --from postgres://user:pass@localhost:5432/tpch
Building customer collection...
Building primary keys...
Building foreign keys...
Ingesting data for table customer...  10 rows done.

Finally, generate data into another instance of Postgres:

$ synth generate tpch --to postgres://user:pass@localhost:5433/tpch

Why Rust

We decided to build Synth from the ground up in Rust. We love Rust, and given the scale of data we wanted synth to generate, it made sense as a first choice. The combination of memory safety, performance, expressiveness and a great community made it a no-brainer and we've never looked back!

Get in touch

If you would like to learn more, or you would like support for your use-case, feel free to open an issue on GitHub.

If your query is more sensitive, you can email opensource@getsynth.com and we'll happily chat about your usecase.

About Us

The Synth project is backed by OpenQuery. We are a YCombinator backed startup based in London, England. We are passionate about data privacy, developer productivity, and building great tools for software engineers.

Contributing

First of all, we sincerely appreciate all contributions to Synth, large or small so thank you.

See the contributing section for details.

License

Synth is source-available and licensed under the Apache 2.0 License.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --> <!-- prettier-ignore-start --> <!-- markdownlint-disable --> <table> <tbody> <tr> <td align="center"><a href="https://github.com/christoshadjiaslanis"><img src="https://avatars.githubusercontent.com/u/14791384?v=4?s=100" width="100px;" alt="Christos Hadjiaslanis"/><br /><sub><b>Christos Hadjiaslanis</b></sub></a><br /><a href="#blog-christoshadjiaslanis" title="Blogposts">πŸ“</a> <a href="#business-christoshadjiaslanis" title="Business development">πŸ’Ό</a> <a href="https://github.com/shuttle-hq/synth/commits?author=christoshadjiaslanis" title="Code">πŸ’»</a> <a href="#content-christoshadjiaslanis" title="Content">πŸ–‹</a> <a href="#design-christoshadjiaslanis" title="Design">🎨</a> <a href="https://github.com/shuttle-hq/synth/commits?author=christoshadjiaslanis" title="Documentation">πŸ“–</a> <a href="#fundingFinding-christoshadjiaslanis" title="Funding Finding">πŸ”</a> <a href="#ideas-christoshadjiaslanis" title="Ideas, Planning, & Feedback">πŸ€”</a> <a href="#infra-christoshadjiaslanis" title="Infrastructure (Hosting, Build-Tools, etc)">πŸš‡</a> <a href="#maintenance-christoshadjiaslanis" title="Maintenance">🚧</a> <a href="#platform-christoshadjiaslanis" title="Packaging/porting to new platform">πŸ“¦</a> <a href="https://github.com/shuttle-hq/synth/pulls?q=is%3Apr+reviewed-by%3Achristoshadjiaslanis" title="Reviewed Pull Requests">πŸ‘€</a> <a href="#security-christoshadjiaslanis" title="Security">πŸ›‘οΈ</a> <a href="https://github.com/shuttle-hq/synth/commits?author=christoshadjiaslanis" title="Tests">⚠️</a> <a href="#talk-christoshadjiaslanis" title="Talks">πŸ“’</a></td> <td align="center"><a href="https://www.linkedin.com/in/ndaneliya/"><img src="https://avatars.githubusercontent.com/u/12720758?v=4?s=100" width="100px;" alt="Nodar Daneliya"/><br /><sub><b>Nodar Daneliya</b></sub></a><br /><a href="#blog-NodarD" title="Blogposts">πŸ“</a> <a href="#business-NodarD" title="Business development">πŸ’Ό</a> <a href="#content-NodarD" title="Content">πŸ–‹</a> <a href="#design-NodarD" title="Design">🎨</a> <a href="https://github.com/shuttle-hq/synth/commits?author=NodarD" title="Documentation">πŸ“–</a> <a href="#fundingFinding-NodarD" title="Funding Finding">πŸ”</a> <a href="#ideas-NodarD" title="Ideas, Planning, & Feedback">πŸ€”</a></td> <td align="center"><a href="https://llogiq.github.io"><img src="https://avatars.githubusercontent.com/u/4200835?v=4?s=100" width="100px;" alt="llogiq"/><br /><sub><b>llogiq</b></sub></a><br /><a href="#business-llogiq" title="Business development">πŸ’Ό</a> <a href="https://github.com/shuttle-hq/synth/commits?author=llogiq" title="Code">πŸ’»</a> <a href="#content-llogiq" title="Content">πŸ–‹</a> <a href="#ideas-llogiq" title="Ideas, Planning, & Feedback">πŸ€”</a> <a href="#infra-llogiq" title="Infrastructure (Hosting, Build-Tools, etc)">πŸš‡</a> <a href="#maintenance-llogiq" title="Maintenance">🚧</a> <a href="#mentoring-llogiq" title="Mentoring">πŸ§‘β€πŸ«</a> <a href="https://github.com/shuttle-hq/synth/pulls?q=is%3Apr+reviewed-by%3Allogiq" title="Reviewed Pull Requests">πŸ‘€</a> <a href="#security-llogiq" title="Security">πŸ›‘οΈ</a> <a href="https://github.com/shuttle-hq/synth/commits?author=llogiq" title="Tests">⚠️</a></td> <td align="center"><a href="https://github.com/shkurskid"><img src="https://avatars.githubusercontent.com/u/77615792?v=4?s=100" width="100px;" alt="Dmitri Shkurski"/><br /><sub><b>Dmitri Shkurski</b></sub></a><br /><a href="https://github.com/shuttle-hq/synth/commits?author=shkurskid" title="Code">πŸ’»</a></td> <td align="center"><a href="https://github.com/brokad"><img src="https://avatars.githubusercontent.com/u/13315034?v=4?s=100" width="100px;" alt="Damien Broka"/><br /><sub><b>Damien Broka</b></sub></a><br /><a href="#blog-brokad" title="Blogposts">πŸ“</a> <a href="#business-brokad" title="Business development">πŸ’Ό</a> <a href="https://github.com/shuttle-hq/synth/commits?author=brokad" title="Code">πŸ’»</a> <a href="#content-brokad" title="Content">πŸ–‹</a> <a href="#design-brokad" title="Design">🎨</a> <a href="https://github.com/shuttle-hq/synth/commits?author=brokad" title="Documentation">πŸ“–</a> <a href="#fundingFinding-brokad" title="Funding Finding">πŸ”</a> <a href="#ideas-brokad" title="Ideas, Planning, & Feedback">πŸ€”</a> <a href="#infra-brokad" title="Infrastructure (Hosting, Build-Tools, etc)">πŸš‡</a> <a href="#maintenance-brokad" title="Maintenance">🚧</a> <a href="https://github.com/shuttle-hq/synth/pulls?q=is%3Apr+reviewed-by%3Abrokad" title="Reviewed Pull Requests">πŸ‘€</a> <a href="https://github.com/shuttle-hq/synth/commits?author=brokad" title="Tests">⚠️</a></td> <td align="center"><a href="https://github.com/fretz12"><img src="https://avatars.githubusercontent.com/u/41805201?v=4?s=100" width="100px;" alt="fretz12"/><br /><sub><b>fretz12</b></sub></a><br /><a href="#ideas-fretz12" title="Ideas, Planning, & Feedback">πŸ€”</a> <a href="https://github.com/shuttle-hq/synth/commits?author=fretz12" title="Code">πŸ’»</a> <a href="https://github.com/shuttle-hq/synth/commits?author=fretz12" title="Documentation">πŸ“–</a> <a href="https://github.com/shuttle-hq/synth/commits?author=fretz12" title="Tests">⚠️</a></td> <td align="center"><a href="https://github.com/baile320"><img src="https://avatars.githubusercontent.com/u/26841355?v=4?s=100" width="100px;" alt="Tyler Bailey"/><br /><sub><b>Tyler Bailey</b></sub></a><br /><a href="https://github.com/shuttle-hq/synth/commits?author=baile320" title="Code">πŸ’»</a> <a href="https://github.com/shuttle-hq/synth/commits?author=baile320" title="Documentation">πŸ“–</a></td> </tr> <tr> <td align="center"><a href="https://github.com/juniorbassani"><img src="https://avatars.githubusercontent.com/u/47573050?v=4?s=100" width="100px;" alt="JΓΊnior Bassani"/><br /><sub><b>JΓΊnior Bassani</b></sub></a><br /><a href="https://github.com/shuttle-hq/synth/issues?q=author%3Ajuniorbassani" title="Bug reports">πŸ›</a> <a href="https://github.com/shuttle-hq/synth/commits?author=juniorbassani" title="Code">πŸ’»</a></td> <td align="center"><a href="https://danielhofstetter.com"><img src="https://avatars.githubusercontent.com/u/26196?v=4?s=100" width="100px;" alt="Daniel Hofstetter"/><br /><sub><b>Daniel Hofstetter</b></sub></a><br /><a href="https://github.com/shuttle-hq/synth/issues?q=author%3Acakebaker" title="Bug reports">πŸ›</a> <a href="https://github.com/shuttle-hq/synth/commits?author=cakebaker" title="Code">πŸ’»</a></td> <td align="center"><a href="https://github.com/AlexMikhalev"><img src="https://avatars.githubusercontent.com/u/50685?v=4?s=100" width="100px;" alt="Dr Alexander Mikhalev"/><br /><sub><b>Dr Alexander Mikhalev</b></sub></a><br /><a href="#maintenance-AlexMikhalev" title="Maintenance">🚧</a> <a href="https://github.com/shuttle-hq/synth/commits?author=AlexMikhalev" title="Documentation">πŸ“–</a> <a href="https://github.com/shuttle-hq/synth/pulls?q=is%3Apr+reviewed-by%3AAlexMikhalev" title="Reviewed Pull Requests">πŸ‘€</a> <a href="#ideas-AlexMikhalev" title="Ideas, Planning, & Feedback">πŸ€”</a> <a href="https://github.com/shuttle-hq/synth/commits?author=AlexMikhalev" title="Code">πŸ’»</a></td> <td align="center"><a href="https://github.com/iamwacko"><img src="https://avatars.githubusercontent.com/u/101361189?v=4?s=100" width="100px;" alt="s e"/><br /><sub><b>s e</b></sub></a><br /><a href="https://github.com/shuttle-hq/synth/commits?author=iamwacko" title="Code">πŸ’»</a> <a href="https://github.com/shuttle-hq/synth/pulls?q=is%3Apr+reviewed-by%3Aiamwacko" title="Reviewed Pull Requests">πŸ‘€</a> <a href="#infra-iamwacko" title="Infrastructure (Hosting, Build-Tools, etc)">πŸš‡</a> <a href="#platform-iamwacko" title="Packaging/porting to new platform">πŸ“¦</a> <a href="https://github.com/shuttle-hq/synth/issues?q=author%3Aiamwacko" title="Bug reports">πŸ›</a> <a href="#ideas-iamwacko" title="Ideas, Planning, & Feedback">πŸ€”</a> <a href="https://github.com/shuttle-hq/synth/commits?author=iamwacko" title="Tests">⚠️</a> <a href="https://github.com/shuttle-hq/synth/commits?author=iamwacko" title="Documentation">πŸ“–</a> <a href="#maintenance-iamwacko" title="Maintenance">🚧</a></td> </tr> </tbody> </table> <!-- markdownlint-restore --> <!-- prettier-ignore-end --> <!-- ALL-CONTRIBUTORS-LIST:END -->

This project follows the all-contributors specification. Contributions of any kind welcome!