Awesome

Overcast is a tiny command line program designed to make it easy to spin up, configure and manage clusters of machines across multiple cloud providers.

Devops made simple.

Create, reboot and destroy instances on DigitalOcean or locally using Virtualbox.

# Spin up a new Ubuntu 20.04 instance on DigitalOcean:
$ overcast digitalocean create db-01

# Perform a full system upgrade and then install Redis:
$ overcast run db-01 install/core install/redis

Run multiple commands or multiple scripts on any of your instances over SSH. Commands can be run sequentially or in parallel. Run and test your deployment scripts across different providers.

# Create a LAMP stack using bundled install scripts:
$ overcast run lamp-01 install/core install/apache install/mysql install/php

# Run your own scripts using relative or absolute paths:
$ overcast run app-cluster ./my-app/my-install-script /path/to/another-script

# Run sequences of commands and scripts across multiple machines in parallel:
$ overcast run app-* ./script.sh uptime "free -m" "df -h" --parallel

Quickly SSH in to any instance by name.

$ overcast ssh app-01

Push and pull files between your local machine and any of your instances.

$ overcast push app-01 nginx/myapp.conf /etc/nginx/sites-enabled/myapp.conf
$ overcast pull app-01 /var/log/syslog ./my-local-syslog-copy

Overcast is a thin layer on top of your SSH client. It doesn't install or leave anything on the servers you communicate with, so Overcast itself has no real attack surface.

A library of scripts and recipes are bundled to make it easy to deploy a number of common software stacks and applications.

Installation (OS X/Linux)

1. Install Node.js if not already installed.

2. Install Overcast using npm.

   $ npm -g install overcast
   

3. You can now use Overcast from any directory. Running any overcast command from anywhere will create the ~/.overcast config directory if it doesn't already exist. Add your API keys to ~/.overcast/variables.json to use their respective commands, either manually or using the var command:

   $ overcast var set DIGITALOCEAN_API_TOKEN abc123
   

4. To make working with Overcast easier, you can add bash tab completion by adding the following to your .bash_profile:

   # Overcast Tab completion
   _overcast_completions() {
     local cur=${COMP_WORDS[COMP_CWORD]}
     COMPREPLY=($(compgen -W "`overcast completions`" -- "$cur"))
     return 0
   }
   complete -F _overcast_completions overcast
   

Installation (Windows)

Using Overcast on Windows is possible, but unsupported.

Uninstallation

Since Overcast is just a wrapper around SSH, there is nothing on your remote machines to uninstall. To uninstall Overcast from your local machine:

# To remove the Overcast package:
$ npm -g remove overcast
# Optionally delete your Overcast SSH keys and configuration files:
$ rm -rf ~/.overcast

Configuration

Overcast looks for an .overcast directory in the current directory, a parent directory, or ~/.overcast, in that order. This means you can have multiple configurations and treat your server infrastructure like source code.

The command overcast init will create a new configuration in the current directory. The config directory looks like this:

/.overcast
  /keys             # SSH keys, can be your own or auto-generated by overcast
    overcast.key
    overcast.key.pub
  clusters.json     # Cluster/instance definitions (see example.clusters.json)
  variables.json    # API keys, etc (see example.variables.json)

Design Goals

There are many server management frameworks out there already, but they generally involve a complex server-client implementation, a steep learning curve, or a giant, monolithic conceptual framework that requires taking a course to understand.

I wanted something that had little to no learning curve, that did only what you asked it to do on the remote machines and nothing more, that just focused on multi-server provisioning and communication and leaves problems like process/state management and system monitoring to tools designed specifically for those problems.

Example App Recipes

• Discourse
• Ubuntu 14.04 LAMP server
• Overcast Charts

Command Reference

overcast aliases

Usage:
  overcast aliases

Description:
  Return a list of bash aliases for SSHing to your instances.

  To use, add this to your .bash_profile:
    test -f $HOME/.overcast_aliases && source $HOME/.overcast_aliases

  And then create the .overcast_aliases file:
    overcast aliases > $HOME/.overcast_aliases

  Or to automatically refresh aliases in every new terminal window
  (which will add a couple hundred milliseconds to your startup time),
  add this to your .bash_profile:
    overcast aliases > $HOME/.overcast_aliases
    source $HOME/.overcast_aliases

overcast cluster count

Usage:
  overcast cluster count [name]

Description:
  Return the number of instances in a cluster.

Examples:
  $ overcast cluster count db
  > 0
  $ overcast instance create db.01 --cluster db
  > ...
  $ overcast cluster count db
  > 1

overcast cluster add

Usage:
  overcast cluster add [name]

Description:
  Adds a new cluster.

Examples:
  $ overcast cluster add db

overcast cluster rename

Usage:
  overcast cluster rename [name] [new-name]

Description:
  Renames a cluster.

Examples:
  $ overcast cluster rename app-cluster app-cluster-renamed

overcast cluster remove

Usage:
  overcast cluster remove [name]

Description:
  Removes a cluster from the index. If the cluster has any instances
  attached to it, they will be moved to an "orphaned" cluster.

Examples:
  $ overcast cluster remove db

overcast completions

Usage:
  overcast completions

Description:
  Return an array of commands, cluster names, and instance names for use
  in bash tab completion.

  To enable tab completion in bash, add this to your .bash_profile:

  _overcast_completions() {
    local cur=${COMP_WORDS[COMP_CWORD]}
    COMPREPLY=($(compgen -W "`overcast completions`" -- "$cur"))
    return 0
  }
  complete -F _overcast_completions overcast

overcast digitalocean boot

Usage:
  overcast digitalocean boot [name]

Description:
  Boot up an instance if powered off, otherwise do nothing.

overcast digitalocean create

Usage:
  overcast digitalocean create [name] [options...]

Description:
  Creates a new instance on DigitalOcean.

Options:                  Defaults:
  --cluster CLUSTER       default
  --ssh-port PORT         22
  --ssh-key PATH          overcast.key
  --ssh-pub-key PATH      overcast.key.pub
  --region REGION
  --image IMAGE
  --size SIZE
  --backups               false
  --monitoring            false
  --private-networking    false
  --vpc-uuid
  --with-droplet-agent    false

Examples:
  # Match using slugs:
  $ overcast digitalocean create vm-01 --size 2gb --region sfo1

  # Match using IDs or names:
  $ overcast digitalocean create vm-02 --region "London 1" --image 6374128

overcast digitalocean destroy

Usage:
  overcast digitalocean destroy [name] [options...]

Description:
  Destroys a DigitalOcean droplet and removes it from your account.
  Using --force overrides the confirm dialog.

Options:     Defaults:
  --force    false

Examples:
  $ overcast digitalocean destroy vm-01

overcast digitalocean images

Usage:
  overcast digitalocean images

Description:
  List all images, including snapshots.

overcast digitalocean instances

Usage:
  overcast digitalocean instances

Description:
  List all instances in your account.

overcast digitalocean reboot

Usage:
  overcast digitalocean reboot [name]

Description:
  Reboot an instance using the provider API.

overcast digitalocean regions

Usage:
  overcast digitalocean regions

Description:
  List all available regions.

overcast digitalocean rebuild

Usage:
  overcast digitalocean rebuild [name] [image]

Description:
  Rebuilds an existing instance on DigitalOcean, preserving the IP address.
  [image] can be image ID, name or slug.

Examples:
  # Rebuild an instance using a readymade image:
  $ overcast digitalocean rebuild vm-01 ubuntu-14-04-x64

  # Rebuild an instance using a snapshot:
  $ overcast digitalocean rebuild vm-01 "vm-01 backup"

overcast digitalocean resize

Usage:
  overcast digitalocean resize [name] [size] [options...]

Description:
  Shutdown, resize, and reboot a DigitalOcean instance.
  [size] must be a valid size slug.
  If the --skip-boot flag is used, the instance will stay powered off.

Options:         Defaults:
  --skip-boot    false

Examples:
  # Resize an instance to 2gb:
  $ overcast digitalocean resize vm-01 2gb

overcast digitalocean snapshot

Usage:
  overcast digitalocean snapshot [name] [snapshot-name]

Description:
  Creates a named snapshot of a droplet. This will reboot the instance.

Examples:
  $ overcast digitalocean snapshot vm-01 vm-01-snapshot

overcast digitalocean snapshots

Usage:
  overcast digitalocean snapshots

Description:
  List all available snapshots in your account.

overcast digitalocean shutdown

Usage:
  overcast digitalocean shutdown [name]

Description:
  Shut down an instance using the provider API.

overcast digitalocean sizes

Usage:
  overcast digitalocean sizes

Description:
  List all available instance sizes.

overcast digitalocean sync

Usage:
  overcast digitalocean sync [name]

Description:
  Fetch and update instance metadata.

overcast expose

Usage:
  overcast expose [instance|cluster|all] [port...] [options]

Description:
  Reset the exposed ports on the instance or cluster using iptables.
  This will fail if you don't include the current SSH port.
  Specifying --allowlist will restrict all ports to the specified address(es).
  These can be individual IPs or CIDR ranges, such as "192.168.0.0/24".

  Expects an Ubuntu server, untested on other distributions.

Options:
  --user USERNAME
  --password PASSWORD
  --allowlist "IP|RANGE"
  --allowlist-PORT "IP|RANGE"

Examples:
  Allow SSH, HTTP and HTTPS connections from anywhere:
  $ overcast expose app 22 80 443

  Allow SSH from anywhere, only allow Redis connections from 1.2.3.4:
  $ overcast expose redis 22 6379 --allowlist-6379 "1.2.3.4"

  Only allow SSH and MySQL connections from 1.2.3.4 or from 5.6.7.xxx:
  $ overcast expose mysql 22 3306 --allowlist "1.2.3.4 5.6.7.0/24"

overcast exposed

Usage:
  overcast exposed [instance|cluster|all]

Description:
  List the exposed ports on the instance or cluster.
  Expects an Ubuntu server, untested on other distributions.

Options:
  --user USERNAME
  --password PASSWORD
  --machine-readable, --mr

overcast help

Usage:
  overcast help

Description:
  Provides help about Overcast and specific commands.

overcast info

Usage:
  overcast info
  overcast info [name]

Description:
  Prints the complete .overcast/clusters.json file.
  Optionally display only instances matching [name].

overcast init

Usage:
  overcast init

Description:
  Create an .overcast config directory in the current working directory.
  No action taken if one already exists.

overcast instance get

Usage:
  overcast instance get [instance|cluster|all] [attr...] [options...]

Description:
  Returns the attribute(s) for the instance or cluster, one per line,
  or space-delimited using the --single-line option.
  Deeply nested arrays and objects are supported.
  "origin" is a compound attribute that returns user@ip:ssh-port.

Options:               Defaults:
  --single-line, -s    false

Examples:
  $ overcast instance get app-01 origin
  root@1.2.3.4:22

  $ overcast instance get app-cluster ip
  127.0.0.1
  127.0.0.2
  127.0.0.3

  $ overcast instance get app-01 digitalocean.image.id
  103510828

overcast instance add

Usage:
  overcast instance add [name] [ip] [options...]

Description:
  Adds an existing instance to a cluster.

Options:                 Defaults:
  --cluster CLUSTER      default
  --ssh-port PORT        22
  --ssh-key PATH         overcast.key
  --user USERNAME        root
  --password PASSWORD

Examples:
  $ overcast instance add app.01 127.0.0.1 --cluster app \
      --ssh-port 22222 --ssh-key $HOME/.ssh/id_rsa

overcast instance list

Usage:
  overcast instance list [cluster...]

Description:
  Returns all instance names, one per line.
  Optionally limit to one or more clusters.

Examples:
  $ overcast instance list
  $ overcast instance list app-cluster db-cluster

overcast instance remove

Usage:
  overcast instance remove [name]

Description:
  Removes an instance from the index.
  The server itself is not affected by this action.

Examples:
  $ overcast instance remove app-01

overcast instance update

Usage:
  overcast instance update [instance|cluster|all] [options...]

Description:
  Update any instance property. Specifying --cluster will move the instance
  to that cluster. Specifying --name will rename the instance.

Options:
  --name NAME
  --cluster CLUSTER
  --ip IP
  --ssh-port PORT
  --ssh-key PATH
  --user USERNAME
  --password PASSWORD

Examples:
  # Update the user and ssh-key of an instance:
  $ overcast instance update app.01 --user myuser --ssh-key /path/to/key

  # Update ssh-port of a cluster:
  $ overcast instance update app-cluster --ssh-port 22222

overcast list

Usage:
  overcast list

Description:
  List your cluster and instance definitions.

overcast ping

Usage:
  overcast ping [instance|cluster|all] [options]

Description:
  Display the average ping time for an instance or cluster.

Options:             Defaults:
  --count N, -c N    3

Examples:
  $ overcast ping app-01
  $ overcast ping db --count 5

overcast port

Usage:
  overcast port [instance|cluster|all] [port]

Description:
  Change the SSH port for an instance or a cluster. This command fails
  if the new port has not been previously opened by iptables.
  See also the "expose" and "exposed" commands.

Examples:
  # Expose only necessary ports:
  $ overcast expose vm-01 22 55522 80 443

  # Update SSH port from 22 to 55522:
  $ overcast port vm-01 55522

  # Close port 22:
  $ overcast expose vm-01 55522 80 443

overcast pull

Usage:
  overcast pull [instance|cluster|all] [source] [dest] [options...]

Description:
  Pull a file or directory from an instance or cluster using scp by default,
  or using rsync if the --rsync flag is used. Source is absolute or relative
  to the home directory. Destination can be absolute or relative to the
  .overcast/files directory. Any reference to {instance} in the destination
  will be replaced with the instance name.

Options:                 Defaults:
  --rsync                false
  --user USERNAME
  --password PASSWORD

Examples:
  Assuming instances "app.01" and "app.02", this will expand to:
    - .overcast/files/app.01.bashrc
    - .overcast/files/app.02.bashrc
  $ overcast pull app .bashrc {instance}.bashrc

overcast push

Usage:
  overcast push [instance|cluster|all] [source] [dest] [options...]

Description:
  Push a file or directory to an instance or cluster using scp by default,
  or rsync if the --rsync flag is used. Source can be absolute or relative
  to the .overcast/files directory. Destination can be absolute or relative
  to the home directory. Any reference to {instance} in the source will be
  replaced with the instance name. The --exclude flag only works with rsync.

Options:                         Defaults:
  --rsync                        false
  --user USERNAME
  --password PASSWORD
  --exclude FILE_OR_DIRECTORY

Examples:
  Assuming instances "app.01" and "app.02", this will expand to:
    - .overcast/files/app.01.bashrc
    - .overcast/files/app.02.bashrc
  $ overcast push app {instance}.bashrc .bashrc

overcast run

Usage:
  overcast run [instance|cluster|all] [command|file...]

Description:
  Execute commands or script files on an instance or cluster over SSH.
  Commands will execute sequentially unless the --parallel flag is used.
  An error will stop execution unless the --continueOnError flag is used.
  Script files can be either absolute or relative path.

Options:                         Defaults:
  --env "KEY=VAL KEY='1 2 3'"
  --user USERNAME
  --password PASSWORD
  --ssh-key PATH
  --ssh-args ARGS
  --continueOnError              false
  --machine-readable, --mr       false
  --parallel, -p                 false
  --shell-command "COMMAND"      bash -s
  --only-once                    false

Examples:
  # Run arbirary commands and files in sequence across all instances:
  $ overcast run all uptime "free -m" "df -h" /path/to/my/script

  # Setting environment variables:
  $ overcast run app --env "foo='bar bar' testing=123" env

  # Use machine-readable output (no server prefix):
  $ overcast run app-01 uptime --mr

  # Run bundled and custom scripts in sequence:
  $ overcast run db-* install/core install/redis ./my/install/script

  # Pass along arbitrary SSH arguments, for example to force a pseudo-tty:
  $ overcast run all /my/install/script --ssh-args "-tt"

overcast scriptvar

Usage:
  overcast scriptvar [instance|cluster|all] [filename] [key] [value]

Description:
  Set a named variable in a remote file on an instance or cluster.
  Expects a shell variable format, for example MY_VAR_NAME="my_value"

Options:                      Defaults:
  --user USERNAME
  --password PASSWORD
  --continueOnError           false
  --machine-readable, --mr    false
  --parallel, -p              false

Examples:
  $ overcast scriptvar app-01 /path/to/file.sh MY_API_TOKEN abc123

overcast slack

Usage:
  overcast slack [message] [options...]

Description:
  Sends a message to a Slack channel.
  Requires a SLACK_WEBHOOK_URL property to be set in variables.json.
  You can set that with the following command:
  overcast var set SLACK_WEBHOOK_URL https://foo.slack.com/blah

Options:                Defaults:
  --channel NAME        #alerts
  --icon-emoji EMOJI    :cloud:
  --icon-url URL
  --user NAME           Overcast
  --KEY VALUE

Examples:
  $ overcast slack "Deploy completed." --icon-emoji ":satelite:"
  $ overcast slack "Server stats" --channel "#general" --cpu "0.54 0.14 0.09"

overcast ssh

Usage:
  overcast ssh [instance] [options...]

Description:
  Opens an interactive SSH connection to an instance.

Options:
  --user USERNAME
  --password PASSWORD
  --ssh-key PATH

Examples:
  $ overcast ssh instance-01
  # To use a personal username and key in variables.json:
  $ overcast vars set OVERCAST_SSH_USER my-username
  $ overcast vars set OVERCAST_SSH_KEY /path/to/my.key
  $ overcast ssh instance-01 # will use the above variables to attempt a connection

overcast sshkey create

Usage:
  overcast sshkey create [name]

Description:
  Creates a new SSH key in the current .overcast config.

Examples:
  $ overcast sshkey create myKeyName
  New SSH key "myKeyName" created.
   - /path/to/.overcast/keys/myKeyName.key
   - /path/to/.overcast/keys/myKeyName.key.pub

overcast sshkey delete

Usage:
  overcast sshkey delete [name]

Description:
  Deletes SSH public/private key files from the current .overcast config.

Examples:
  $ overcast sshkey delete myKeyName
  SSH key "myKeyName" deleted.

overcast sshkey get

Usage:
  overcast sshkey get [name] [option]

Description:
  Display the requested SSH key data or path from the current .overcast config.
  Defaults to displaying the public key data if no option found.

Options:
  --public-data
  --private-data
  --public-path
  --private-path

Examples:
  $ overcast sshkey get myKeyName
  [public key data]
  $ overcast sshkey get myKeyName --private-data
  [private key data]

overcast sshkey list

Usage:
  overcast sshkey list

Description:
  List the found SSH key names in the current .overcast config.

Examples:
  $ overcast sshkey list
  myKeyName
  overcast

overcast sshkey push

Usage:
  overcast sshkey push [instance|cluster|all] [name|path] [options...]

Description:
  Push a public SSH key to an instance or cluster. Accepts a key name,
  filename, or full path. This will overwrite the existing authorized_keys
  file, unless you use --append.

Options:             Defaults:
  --user USERNAME
  --append, -a       false

Examples:
  # Generate new SSH key pair:
  $ overcast sshkey create newKey

  # Push public key to instance, update instance config to use private key:
  $ overcast sshkey push vm-01 newKey
  $ overcast instance update vm-01 --ssh-key newKey.key

  # Same as above but using key path instead of key name:
  $ overcast sshkey push vm-02 "~/.ssh/id_rsa.pub"
  $ overcast instance update vm-02 --ssh-key "~/.ssh/id_rsa"

  # Push public key to instance using arbitrary user:
  $ overcast sshkey push vm-03 newKey --user myOtherUser

  # Append public key to authorized_keys instead of overwriting:
  $ overcast sshkey push vm-04 newKey --append

overcast tunnel

Usage:
  overcast tunnel [instance] [local-port((:hostname):remote-port)...]

Description:
  Opens an SSH tunnel to the port(s) specified.
  If only one port is specified, assume the same port for local/remote.
  If no remote host is specified, assume the remote host itself (127.0.0.1).
  Multiple tunnels can be opened over a single connection.

Options:
  --user USERNAME
  --password PASSWORD
  --ssh-key PATH

Examples:
  # Tunnel local 5984 to remote 5984
  $ overcast tunnel app-01 5984

  # Tunnel local 8000 to remote 5984, local 8001 to remote 3000
  $ overcast tunnel app-01 8000:5984 8001:3000

  # Tunnel local 3000 to otherhost.com:4000
  $ overcast tunnel app-01 3000:otherhost.com:4000

overcast vars list

Usage:
  overcast vars list

Description:
  List all variables in your current config.

overcast vars set

Usage:
  overcast vars set [name] [value]

Description:
  Set a variable in your current config.

Examples:
  $ overcast vars set AWS_KEY myawskey12345
  $ overcast vars set MY_CUSTOM_VARIABLE_NAME foo

overcast vars get

Usage:
  overcast vars get [name]

Description:
  Get a variable from your current config.

Examples:
  $ overcast vars get AWS_KEY
  > myawskey12345

  $ overcast vars get MY_CUSTOM_VARIABLE_NAME
  > foo

overcast vars delete

Usage:
  overcast vars delete [name]

Description:
  Delete a variable from your current config.

Examples:
  $ overcast vars delete MY_CUSTOM_VARIABLE_NAME

overcast virtualbox boot

Usage:
  overcast virtualbox boot [name]

Description:
  Boot up a Virtualbox instance.

overcast virtualbox create

Usage:
  overcast virtualbox create [name] [options...]

Description:
  Creates a new Virtualbox instance.

Options:                Defaults:
  --cluster CLUSTER     default
  --cpus COUNT          1
  --image NAME          trusty64
  --ram MB              512
  --ip ADDRESS          192.168.22.10
  --ssh-key PATH        overcast.key
  --ssh-pub-key PATH    overcast.key.pub

Examples:
  $ overcast virtualbox create vm-01
  $ overcast virtualbox create vm-02 --ram 1024 --image precise64

overcast virtualbox destroy

Usage:
  overcast virtualbox destroy [name] [options...]

Description:
  Destroys a Virtualbox instance.
  Using --force overrides the confirm dialog.

Options:     Defaults:
  --force    false

Examples:
  $ overcast virtualbox destroy vm-01

overcast virtualbox reboot

Usage:
  overcast virtualbox reboot [name]

Description:
  Reboots a Virtualbox instance.

overcast virtualbox shutdown

Usage:
  overcast virtualbox shutdown [name]

Description:
  Shut down a Virtualbox instance.

overcast wait

Usage:
  overcast wait [seconds]

Description:
  Show a progress bar for a specified number of seconds.

Examples:
  $ overcast wait 30

Running the Tests

npm test

Upgrading Overcast

npm -g update overcast

Configuration files are left alone during an upgrade.

Contributing

Contributions are welcome. If you've got an idea for a feature or found a bug, please open an issue. If you're a developer and want to help improve Overcast, open a pull request with your changes.

Hacking Overcast

git clone git@github.com:andrewchilds/overcast.git
cd overcast

# To use local development version:
node overcast [args...]

# To use published version:
overcast [args...]

Roadmap

• Standardize Recipes
• Tagging
• Google Compute Engine support
• More comprehensive script/recipe library
• More test coverage

License

MIT. Copyright © 2014-2022 Andrew Childs