Awesome

Grai Actions

Shared Fields

All actions share some common fields listed below.

Authentication

You must provider either api-key or grai-user and grai-password. If you're account is associated with multiple workspaces and you're using username/password authentication you must also provide your desired workspace.

Other Parameters

Notes and Caveats

Github Authentication

By default we use a github-token provided by your repository to write comments back to your PR with test results. In some cases, such as when the pull request is coming from a forked repository, the default token will not have write permissions. If this is the case, you'll receive an error message in the workflow indicating such. There are a few ways you can resolve the issue but you should first check your repository action settings under Settings -> Actions -> General aren't blocking workflows from running.

Some alternatives include.

Explicit Workflow Permissions

GitHub has provided helpful documentation to provide explicit permissions for your workflows. Make sure the Grai Action has, at minimum, write permissions for pull-request and issues. You can set this at the job level by adding a permission key in your workflow. e.g.

jobs:
  my-grai-action:
    runs-on: ubuntu-latest

    permissions:
      issues: write
      pull-requests: write

Personal Access Tokens

You can also use personal access tokens or PAT's in place of the default github-token. You'll need to create a token following the linked instructions but make sure to store it in your repository secrets Settings -> Secrets and variables -> Actions -> New Repository Secret. If you were to create a secret called MY_PAT you would pass it into your grai action job as

jobs:
  my-grai-action:
    runs-on: ubuntu-latest

  steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Run Grai Action
      uses: grai-io/grai-actions/redshift@master
      with:
        github-token: ${{ secrets.MY_PAT }}

Snowflake

The Snowflake action depends on Snowflake's python connector library. You can find complete documentation about the library in the Snowflake docs here with more detail about the connector here.

Fields

Example

on:
  - pull_request
name: Snowflake
jobs:
  test_snowflake:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/snowflake@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          db-user: my-user
          db-password: my-password
          account: my-account
          warehouse: my-warehouse

Redshift

The Redshift action depends on Amazon's python connector library. You can find complete documentation about the library in the AWS docs here.

Fields

Example

on:
  - pull_request
name: Redshift
jobs:
  test_redshift:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/redshift@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          db-host: redshift-cluster-1.abc123xyz789.us-east-1.redshift.amazonaws.com
          db-port: '5439'
          db-database-name: dev
          db-user: admin
          db-password: password

MySQL

The MySQL action depends on the python mysql library. You can find complete documentation about the library here.

Fields

Example

on:
  - pull_request
name: MySQL
jobs:
  test_mysql:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/mysql@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          db-host: dev.mysql.com
          db-port: '3306'
          db-database-name: my_db
          db-user: my_user
          db-password: my_password

Flat File

The flat file action reads a flat file like csv, parquet, or feather inside of your github project to perform tests and update your grai instance. Because of this, it's critical your file is up to date on each pull request.

Make sure to include an - uses: actions/checkout@v3 step in your workflow so that your repo code is available.

Fields

Example

on:
  - pull_request
name: Flat File
jobs:
  test_flat-file:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/flat-file@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          file: ./tests/flat-file/low-numbers.csv

BigQuery

The BigQuery action depends on Google's python BigQuery library. More information can be found about specific connection credentials in Google's documentation here.

Fields

Example

on:
  - pull_request
name: BigQuery
jobs:
  test_bigquery:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/bigquery@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          project: my-bigquery-project
          dataset: my-bigquery-dataset
          credentials: '{ "type": "service_account", "project_id": "demo", "private_key_id":
            "your_private_key_id", "private_key": "your_private_key", "client_email":
            "your@email.iam.gserviceaccount.com", "client_id": "your_client_id", "auth_uri":
            "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token",
            "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
            "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/you%40email.iam.gserviceaccount.com"
            }'

Postgres

The Postgres action depends on the python psycopg2 library. You can find complete documentation about the library here.

Fields

Example

on:
  - pull_request
name: PostgreSQL
jobs:
  test_postgres:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/postgres@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          db-host: prod.db.com
          db-port: '5432'
          db-database-name: my_database
          db-user: my_user
          db-password: my_password

dbt

The dbt action reads a manifest.json file inside of your github project to perform tests and update your grai instance. Because of this, it's critical your manifest.json file is up to date on each pull request. One way to do this is to perform dbt build as part of your CI action but there are multiple ways to generate an up-to-date manifest file.

• More information about manifest.json files can be found here.
• Make sure to include an - uses: actions/checkout@v3 step in your workflow so that your repo code is available.

Fields

Example

on:
  - pull_request
name: dbt
jobs:
  test_dbt:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/dbt@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          manifest-file: ./tests/dbt/manifest.json

Fivetran

The Fivetran Action relies upon access to Fivetran's API endpoint. This endpoint is configurable if you have a non-standard implementation but should generally be left alone.

Authentication with their services will require an API key and secret but you can find more documentation about generating these values here.

Fields

The namespace field in the Fivetran Action works slightly differently than other action. It is used as a default namespace for all connections not specified in the namespace_map. You can find more information about that below.

Namespace Map

Each Fivetran connection has a connector id and synchronizes from a source to a sync. For example, a sync from your production database to data warehouse would have an associated connector id.

Because the Fivetran Action synchronizes from all of your Fivetran connections it uses the namespace_map value to know which connectors belong to which Grai namespaces. The namespace map should be a JSON string with the Grai namespace for each source and destination of each connector id e.g.

{
    "<connector_id>": {
        "source": "<source_namespace>",
        "destination", "<destination_namespace>"
    }
}

You can find connector id's for all of your Fivetran connections in the API

Example

on:
  - pull_request
name: Fivetran
jobs:
  test_fivetran:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/fivetran@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          fivetran-api-key: hHqP5c2nIY0B6fpa
          fivetran-api-secret: 1234567890abcdef1234567890abcdef
          namespace-map: '{"operative_combination": {"source": "source_namespace",
            "destination": "destination_namespace"}}'

Microsoft SQL Server

The SQL Server action depends on the python pyodbc library. You can find complete documentation about the library here.

There are a variety of ways to configure a pyodbc connection depending on your security implementation. A standard connection would consist of a host, port, database name, user, and password.

Fields

Example

on:
  - pull_request
name: SQL Server
jobs:
  test_mssql:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Run Grai Action
        uses: grai-io/grai-actions/mssql@master
        with:
          namespace: my_apps_grai_namespace
          api-key: my_grai_api_key
          action: tests
          source-name: prod-db
          grai-api-url: https://api.grai.io
          db-user: sa
          db-password: sa_password
          server_connection_string: tcp:myserver,1433
          trust_server_certificate: 'true'