Awesome
dbt Cloud Cancel Running CI Job Runs Action
NOTE: This Github action is now obsolete now that this functionality is avalible nativelty within dbt Cloud CI Jobs
- This repo will no longer be updated or supported
This action checks if a dbt Cloud CI job (based on job ID) is running after a new or updated PR commit is made. If there is a job running or there are job runs that are queued - this action will cancel these running and queued runs since they are likely stale as a new commit to the PR has been made. This allows to always have the newest code commit running in the CI job without having to wait for the stale job runs to finish.
Some scenarios where this is useful:
- You have longer running CI jobs in your dbt Cloud project and you don't want new commits to get queued when they are replacing stale code that is still running in a previous CI job
- You have a large team where there is constant commits/PRs against the code base and you always want the latest job to be run without waiting for older jobs to finish
To view a Loom video of this action being used, click here
Inputs
Required Inputs
dbt_cloud_token
- dbt Cloud API tokendbt_cloud_account_id
- dbt Cloud Account IDdbt_cloud_job_id
- dbt Cloud Job ID
Optional Inputs
only_cancel_run_if_commit_is_using_pr_branch
- A flag that can be set totrue
orfalse
- When this flag is set to
true
, CI jobs will only get cancelled if the branch that the CI job is running off of is the same as the branch the PR is using - Example of using this: If someone kicks off a CI run based off the branch
branch_1
and then someone else kicks off a CI run based on the branchbranch_2
- When this flag is set to
true
the GitHub Action triggered via the PR onbranch_2
will not cancel the CI run based onbranch_1
- When this flag is set to
false
(or not set at all) the GitHub Action will cancel the run based onbranch_1
as it doesn't care if the branch is related to the newest PR or not - the behavior is to always get the lastest CI job running regardless of branch
- When this flag is set to
- When this flag is set to
true
it requires that the inputgithub_pr_number
be configured to${{ github.event.number }}
- When this flag is set to
github_pr_number
- The number of the pull request in GitHub, this is used when limiting cancelling of jobs to just a given PR that is being worked on This is only needed ifonly_cancel_run_if_commit_is_using_pr_branch
is set totrue
dbt_cloud_host
- the URL of the dbt cloud account with, by defaultcloud.getdbt.com
is usedmax_runs
- the number of runs to look back and cancel for a given dbt Cloud job, by default10
is usedonly_cancel_queued_starting_run
- A flag that can be set totrue
orfalse
- When this flag is set to
true
, the action will only cancel dbt Cloud job runs that are in the stateQueued
orStarting
- The purpose of this flag is to prevent the action from cancelling jobs that are already running when a new CI job is kicked off. Some use cases require this as they don't want to cancel a job that is halfway completed
- When this flag is set to
cancel_runs_based_on_schema_override
- A flag that can be set totrue
orfalse
- When this and
only_cancel_run_if_commit_is_using_pr_branch
are both set totrue
, this action will find the associated PR number of existing dbt runs based on theirschema_override
. Specifically it assumes theschema_override
will be of the formdbt_cloud_pr_{dbt_cloud_job_id}_{github_pr_number}
, which matches that of dbt Cloud CI. - The purpose of this flag is to enable canceling stale CI runs even if they weren't triggered directly by the dbt Cloud <> GitHub integration. For example, teams triggering CI jobs via GitHub Actions (or any other CI/CD tool) can still utilize this action, as long as the
schema_override
parameter is configured properly in the CI job.
- When this and
It's recommend to pass sensitive variables as GitHub secrets. Example article on how to use Github Action secrets
Outputs
cancelled_jobs_flag
- A returned flag that is outputted asTrue
if running/queued job runs were cancelled in order to kick off a CI job for the latest commit. ReturnsFalse
if no job runs were running/queued and therefore didn't need to be cancelled.cancelled_dbt_cloud_job_runs
- A list of dbt Cloud run IDs for the given job that were cancelled in order to kick off a CI job for the latest commit. (e.g.[85660002, 85660002]
). This is useful for logging.cancelled_dbt_cloud_job_runs_markdown
- Pre-Scripted markdown containing info on the cancelled jobs, can be used in PR comments. Example of a scenario where 4 job runs where cancelled this is the markdown output:**The following dbt Cloud job runs were cancelled to free up the queue for the new CI job on the current PR:** - Run **85519539** was cancelled at **2022-09-29 23:32:44 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519539/) - Run **85519497** was cancelled at **2022-09-29 23:32:45 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519497/) - Run **85519494** was cancelled at **2022-09-29 23:32:46 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519494/) - Run **85519490** was cancelled at **2022-09-29 23:32:46 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519490/)
Creating a workflow
A workflow using the only_cancel_run_if_commit_is_using_pr_branch
flag
# This is a basic workflow to show using this action
# name of the workflow
name: Cancel running slim ci job runs if new commit is made to the same pull request triggering a new run
# Controls when the workflow will run
on:
pull_request:
branches: [ "main" ]
# Allows you to run this workflow manually from the Actions tab if needed
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "cancel_running_slim_ci_jobs"
cancel_running_slim_ci_jobs:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# running the step to cancel another other CI job runs that are running except the latest, only for the given PR
- name: Cancel other CI runs that are running for the given PR except the latest run
id: cancel_stale_ci_runs
uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
with:
dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
dbt_cloud_account_id: 12345
dbt_cloud_job_id: 130247
only_cancel_run_if_commit_is_using_pr_branch: true
github_pr_number: ${{ github.event.number }}
max_runs: "50"
A workflow not using the only_cancel_run_if_commit_is_using_pr_branch
flag
# This is a basic workflow to show using this action
# name of the workflow
name: Cancel all other running slim ci job runs if new commit is made
# Controls when the workflow will run
on:
pull_request:
branches: [ "main" ]
# Allows you to run this workflow manually from the Actions tab if needed
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "cancel_running_slim_ci_jobs"
cancel_running_slim_ci_jobs:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# running the step to cancel another other CI job runs that are running except the latest
- name: Cancel all other ci runs that are running except the latest run
id: cancel_stale_ci_runs
uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
with:
dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
dbt_cloud_account_id: 12345
dbt_cloud_job_id: 130247
Examples of using this Github action with other workflows
<br>Example using workflow to log the cancelled job run IDs:
# This is a basic workflow to show using this action
# name of the workflow
name: Cancel running slim ci job runs if new commit is made to the same pull request triggering a new run
# Controls when the workflow will run
on:
pull_request:
branches: [ "main" ]
# Allows you to run this workflow manually from the Actions tab if needed
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "cancel_running_slim_ci_jobs"
cancel_running_slim_ci_jobs:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# running the step to cancel another other CI job runs that are running except the latest, only for the given PR
- name: Cancel other CI runs that are running for the given PR except the latest run
id: cancel_stale_ci_runs
uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
with:
dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
dbt_cloud_account_id: 12345
dbt_cloud_job_id: 130247
only_cancel_run_if_commit_is_using_pr_branch: true
github_pr_number: ${{ github.event.number }}
max_runs: "50"
# logging if there was a job run(s) cancelled or not
- name: Logging if there was a CI run that was cancelled
if: steps.cancel_stale_ci_runs.outputs.cancelled_jobs_flag == 'True'
run: |
echo "A dbt Cloud CI job run(s) was cancelled due to a new CI run being triggered"
echo "The following job runs were cancelled ${{ steps.cancel_stale_ci_runs.outputs.cancelled_dbt_cloud_job_runs }}"
<br>
Example using workflow to cancel job runs and then post information about the cancelled job runs in a PR comment
This workflow will produce a PR comment that looks like this when job runs are cancelled:
# This is a basic workflow to show using this action
# name of the workflow
name: Cancel running slim ci job runs if new commit is made to the same pull request triggering a new run
# Controls when the workflow will run
on:
pull_request:
branches: [ "main" ]
# Allows you to run this workflow manually from the Actions tab if needed
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "cancel_running_slim_ci_jobs"
cancel_running_slim_ci_jobs:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# running the step to cancel another other CI job runs that are running except the latest, only for the given PR
- name: Cancel other CI runs that are running for the given PR except the latest run
id: cancel_stale_ci_runs
uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
with:
dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
dbt_cloud_account_id: 12345
dbt_cloud_job_id: 130247
only_cancel_run_if_commit_is_using_pr_branch: true
github_pr_number: ${{ github.event.number }}
max_runs: "50"
# logging if there was a job run(s) cancelled or not
- name: Logging if there was a CI run that was cancelled
if: steps.cancel_stale_ci_runs.outputs.cancelled_jobs_flag == 'True'
run: |
echo "A dbt Cloud CI job run(s) was cancelled due to a new CI run being triggered"
echo "The following job runs were cancelled ${{ steps.cancel_stale_ci_runs.outputs.cancelled_dbt_cloud_job_runs }}"
# if there a job run(s) was cancelled, we grab the outputted markdown and put it into a PR comment for logging
- name: PR comment with Job Run Cancelation Information
uses: mshick/add-pr-comment@v1
if: steps.cancel_stale_ci_runs.outputs.cancelled_jobs_flag == 'True'
with:
message: |
${{ steps.cancel_stale_ci_runs.outputs.cancelled_dbt_cloud_job_runs_markdown }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
repo-token-user-login: 'github-actions[bot]'
allow-repeats: false # This is the default