Awesome
Retry
⏰ Rerun a command until it eventually succeeds, or doesn't!
Installation
Prebuilt binaries for several architectures can be found attached to any of the available releases.
For Linux:
wget https://github.com/joshdk/retry/releases/download/v1.4.0/retry-linux-amd64.tar.gz
tar -xf retry-linux-amd64.tar.gz
sudo install retry /usr/bin/retry
For Mac:
brew tap joshdk/tap
brew install joshdk/tap/retry
A development version can also be built directly from this repository. Requires that you already have a functional Go toolchain installed.
go install github.com/joshdk/retry@master
Motivations
I kept seeing folks write bespoke code to retry commands that were either flaky, or took time to succeed. This usually manifested as some sort of loop in bash, with a counter, and a return code check.
Searching around, this doesn't seem to be an isolated problem, which has an even larger number of bespoke solutions. Take for example this handful of Stack Overflow threads:
- How to retry a command in Bash?
- Retry a Bash command with timeout
- How do I write a retry logic in script to keep retrying to run it upto 5 times?
These are perfectly legitimate questions, with many reasonable answers. The downside is that the solutions were usually specific to the question asked, and not always applicable to the broader problem.
This tool is an attempt to solve that broader problem. ⏰
Usage
Help!
Usage: retry [flags] command|url
-attempts int
maximum number of attempts (default 3)
-backoff
use exponential backoff when sleeping
-consecutive int
required number of back to back successes
-delay duration
initial delay period before tasks are run
-invert
wait for task to fail rather than succeed
-jitter duration
time range randomly added to sleep
-max-time duration
maximum total time (default 1m0s)
-quiet
silence all output
-sleep duration
time to sleep between attempts (default 5s)
-task-time duration
maximum time for a single attempt
-version
print the version "v1.4.0" and exit
Running a command
Retry will run a given command repeatedly, until it is deemed an overall success of failure. The conditions and limits for what determine success/failure can be tuned with command line flags.
As a special case, if a URL is given, retry will GET that URL and check for a 200 OK to be returned.
Limit attempts
The -attempts
flag limits the maximum number of times a command can be run. A value of 0 allows unlimited attempts.
Run
cat kubeconfig.yml
a maximum of 3 times, or less if the command succeeds earlier:$ retry -attempts=3 cat kubeconfig.yml
Limit task time
The -task-time
flag limits the maximum time that a command can run for. A value of 0 allows unlimited time.
Run
wget https://example.com
, but limit the command to only run for a maximum of 15 seconds.$ retry -task-time=15s wget https://example.com
Limit overall time
The -max-time
flag limits the maximum total time that retry
will run for. A value of 0 allows unlimited time.
GET
https://example.com
repeatedly, but stop running after a total of 60 seconds.$ retry -max-time=60s https://example.com
Initial delay
The -delay
flag inserts a one-time delay before initial starting to run commands.
Run
wget https://example.com
, but start only after initially sleeping for 15 seconds.$ retry -delay=15s wget https://example.com
Sleep between attempts
The -sleep
flag inserts a timed delay between command runs.
Run
cat kubeconfig.yml
, but sleep for 15 seconds between runs.$ retry -sleep=15s cat kubeconfig.yml
Exponential backoff
The -backoff
flag is used with -sleep
, and will double the time delay between failed runs. Delay is reset after a successful run.
Run
wget https://example.com
, sleeping for 15 seconds after the first failure, 30 seconds after the second failure, 1 minute after the third failure, etc...$ retry -sleep=15s -backoff wget https://example.com
Invert status
The -invert
flag is used to flip a task's failure status. Successful task runs will become failures, and vice versa. Useful for when you want to retry a command until it fails.
Run
curl https://example.com/health
, a maximum of 20 times, until it becomes unresponsive.$ retry -attempts=20 -invert curl https://example.com/health
Random jitter
The -jitter
flag adds a random time range to the sleep duration. Jitter added on top of exponential backoff.
Run
cat kubeconfig.yml
, sleep for 15 seconds minimum, plus a random 0-10 seconds between each run.$ retry -sleep=15s -jitter=10s cat kubeconfig.yml
Consecutive successes
The -consecutive
flag requires a number of successful command runs to occur in a row in order to be considered successful. Useful for health checking a service that is inconsistent until if if fully started.
GET
https://example.com
, requiring the command to be successful 3 times in a row.$ retry -consecutive=3 wget https://example.com
Be quiet!
Lastly, the -quiet
flag silences all output (STDOUT and STDERR) from the command. Useful when running retry
inside an if
.
Run
ls -R
, but swallow all output.$ retry -quiet ls -R
Altogether now
Run
wget https://example.com
a maximum of 10 times. Each run can take a maximum of 15 seconds, and a total of 2 minutes. Delay for 15 seconds before starting. Sleep for 5 seconds between failures with exponential backoff. Lastly, require that the command succeeds 3 times in a row.$ retry -attempts=10 -task-time=15s -max-time=2m -delay=15s -sleep=5s -backoff -consecutive=3 wget https://example.com
License
This code is distributed under the MIT License, see LICENSE.txt for more information.