Home

Awesome

Vagrant Google Compute Engine (GCE) Provider

Gem Version

This is a Vagrant plugin that adds an Google Compute Engine (GCE) provider to Vagrant, allowing Vagrant to control and provision instances in GCE.

NOTE: The plugin is currently looking for maintainers, please contact @temikus.

Features

Requirements

Google Cloud Platform Setup

Do the following:

  1. Log in with your Google Account and go to Google Cloud Platform and click on the Try it free button.
  2. Create a new project and remember to record the Project ID
  3. Enable the Google Compute Engine API for your project in the API console. If prompted, review and agree to the terms of service.
  4. Install the Cloud SDK
  5. Run `gcloud auth application-default login to create your credentials. (Alternatively, you may use a service account, see Using a Service Account section).
  6. Add the public SSH key you're going to use to GCE Metadata in Compute -> Compute Engine -> Metadata section of the console, SSH Keys tab. (Read the SSH Support readme section for more information.)

Using a Service Account

The `appplication-default login method is intended to be used for developing code on a local environment - this is typically Vagrant's use-case as well. However, if this is not your use-case, you will want to use a credential not tied to your local environment: a service account.

To use a service account:

  1. While still in the API & Services, go to Credentials subsection, and click Create credentials -> Service account.
  2. Create a Service Account with any name (f.e. vagrant) and grant it a Compute Admin role.
  3. Open the new service account page and click on the Keys tab. Click Add key -> Create new key, choose JSON. Download the JSON private key and save this file in a secure and reliable location.

Then include the private key in your Vagrantfile's provider block as a google_json_key_location attribute:

Vagrant.configure("2") do |config|
  # ... other stuff

  config.vm.provider :google do |google|
    google.google_project_id = "YOUR_GOOGLE_CLOUD_PROJECT_ID"
    google.google_json_key_location = "/path/to/your/private-key.json"
  end
end

Vagrant Setup

Install as a Vagrant plugin:

vagrant plugin install vagrant-google

Usage

Make a Vagrantfile that looks like the following, filling in your information where necessary:

Vagrant.configure("2") do |config|
  config.vm.box = "google/gce"

  config.vm.provider :google do |google, override|
    google.google_project_id = "YOUR_GOOGLE_CLOUD_PROJECT_ID"
    google.image_family = 'ubuntu-2004-lts'

    override.ssh.username = "USERNAME"
    override.ssh.private_key_path = "~/.ssh/id_rsa"
  end

end

Run:

vagrant up --provider=google

This will start the latest version of Ubuntu 20.04 LTS instance in the us-central1-f zone, with an n1-standard-1 machine, and the "default" network within your project. And assuming your SSH information (see below) was filled in properly within your Vagrantfile, SSH and provisioning will work as well.

Note that normally a lot of this boilerplate is encoded within the box file, but the box file used for the quick start, the "google" box, has no preconfigured defaults.

SSH Support

In order for SSH to work properly to the GCE VM, you will first need to add your public key to the GCE metadata service for the desired VM user account. When a VM first boots, a Google-provided daemon is responsible for talking to the internal GCE metadata service and creates local user accounts and their respective ~/.ssh/authorized_keys entries. Most new GCE users will use the Cloud SDK gcloud compute utility when first getting started with GCE. This utility has built in support for creating SSH key pairs, and uploading the public key to the GCE metadata service. By default, gcloud compute creates a key pair named ~/.ssh/google_compute_engine[.pub].

Note that you can use the more standard ~/.ssh/id_rsa[.pub] files, but you will need to manually add your public key to the GCE metadata service so your VMs will pick up the key. Note that the public key is typically prefixed with the username, so that the daemon on the VM adds the public key to the correct user account.

Additionally, you will probably need to add the key and username to override settings in your Vagrantfile like so:

config.vm.provider :google do |google, override|

    #...google provider settings are skipped...

    override.ssh.username = "testuser"
    override.ssh.private_key_path = "~/.ssh/id_rsa"

    #...google provider settings are skipped...

end

See the links below for more help with SSH and GCE VMs.

Box Format

Every provider in Vagrant must introduce a custom box format. This provider introduces google boxes. You can view an example box in example_boxes/. That directory also contains instructions on how to build a box.

The box format is basically just the required metadata.json file along with a Vagrantfile that does default settings for the provider-specific configuration for this provider.

Configuration

This provider exposes quite a few provider-specific configuration options:

Vagrant.configure("2") do |config|
  # ... other stuff

  config.vm.provider :google do |google|
    google.google_project_id = "YOUR_GOOGLE_CLOUD_PROJECT_ID"
    google.google_json_key_location = "/path/to/your/private-key.json"
  end
end

In addition to the above top-level configs, you can use the zone_config method to specify zone-specific overrides within your Vagrantfile. Note that the top-level zone config must always be specified to choose which zone you want to actually use, however. This looks like this:

Vagrant.configure("2") do |config|

  config.vm.box = "google/gce"

  config.vm.provider :google do |google|
    google.google_project_id = "YOUR_GOOGLE_CLOUD_PROJECT_ID"
    google.google_json_key_location = "/path/to/your/private-key.json"

    # Make sure to set this to trigger the zone_config
    google.zone = "us-central1-f"

    google.zone_config "us-central1-f" do |zone1f|
        zone1f.name = "testing-vagrant"
        zone1f.image = "debian-9-stretch-v20211105"
        zone1f.machine_type = "n1-standard-4"
        zone1f.zone = "us-central1-f"
        zone1f.metadata = {'custom' => 'metadata', 'testing' => 'foobarbaz'}
        zone1f.scopes = ['bigquery', 'monitoring', 'https://www.googleapis.com/auth/compute']
        zone1f.tags = ['web', 'app1']
    end
  end
end

The zone-specific configurations will override the top-level configurations when that zone is used. They otherwise inherit the top-level configurations, as you would expect.

There are a few example Vagrantfiles located in the vagrantfile_examples/ directory.

Networks

Networking features in the form of config.vm.network are not supported with vagrant-google, currently. If any of these are specified, Vagrant will emit a warning, but will otherwise boot the GCE machine.

Synced Folders

Since plugin version 2.0, this is implemented via built-in SyncedFolders action. See Vagrant's rsync action documentation for more info.

Automatic shutdown

To save money you may want to ensure you don't forget to shut down your instances when you stop using them.

A very basic solution for this is to use Vagrant's provisioning feature to plan automatic shutdown of the vm after given time after each vagrant up:

# Plan automatic shutdown of machine to prevent unwanted costs
config.vm.provision "auto-shutdown", type: "shell", run: "always",
  inline: "shutdown -P +480" # = 60 minutes * 8 hours

Print external IP

You may want to know your machine's external IP f.e. to put it in your Ansible inventory or open the app you deploy in it in your browser.

To automate printing it IP you can also use the Vagrant's provisioning feature:

# Print the external IP
config.vm.provision "print-ip", type: "shell", run: "always",
  inline: "echo External IP: $(curl -s icanhazip.com)"

Development

To work on the vagrant-google plugin, clone this repository, and use Bundler to get the dependencies:

$ bundle

Once you have the dependencies, verify the unit tests pass with rake:

$ bundle exec rake

If those pass, you're ready to start developing the plugin. You can test the plugin without installing it into your Vagrant environment by just creating a Vagrantfile in the top level of this directory (it is ignored by git), and use bundler to execute Vagrant:

$ bundle exec vagrant up --provider=google

Acceptance testing

Work-in-progress: Acceptance tests are based on vagrant-spec library which is currently under active development so they may occasionally break.

Before you start acceptance tests, you'll need to set the authentication shell variables accordingly:

export GOOGLE_PROJECT_ID="your-google-cloud-project-id"
export GOOGLE_JSON_KEY_LOCATION="/full/path/to/your/private-key.json"

export GOOGLE_SSH_USER="testuser"
export GOOGLE_SSH_KEY_LOCATION="/home/testuser/.ssh/id_rsa"

After, you can run acceptance tests by running the full task in acceptance namespace:

$ bundle exec rake acceptance:full

IMPORTANT NOTES:

Changelog

See CHANGELOG.md

License

Apache 2.0; see LICENSE for details.