Awesome

modal-dbt

This repo gives some code to run dbt jobs/actions using modal which is a serverless application framework. You can test it out by creating a codespace, starting here

deploy the run_dbt stub via:

modal deploy run_dbt.py

You'll then need to add secrets to Modal for database connection credentials (they get passed as env_vars to my profiles.yml). You should add a web endpoint authentication token as we'll use this to authenticate requests to our run_dbt function triggered via a curl request.

You can then authenticate and trigger your modal dbt job application from anywhere, whether that be python invocation in an orchestration tool, or a simple curl in your unix shell.

curl --header "Authorization: Bearer my-auth-token" 'https://trouze--run-dbt-demo-run-dbt.modal.run?repo_url=https://github.com/trouze/dbt-slim-ci.git&dbt_commands=run,-s,stg_customers'