Home

Awesome

Berglas

Build Status GoDoc

Berglas Logo

Berglas is a command line tool and library for storing and retrieving secrets on Google Cloud. Secrets are encrypted with Cloud KMS and stored in Cloud Storage. An interoperable layer also exists with Secret Manager.

Berglas is not an officially supported Google product.

Setup

Prerequisites

  1. Install the Cloud SDK for your operating system. Alternatively, you can run these commands from Cloud Shell, which has the SDK and other popular tools pre-installed.

    If you are running from your local machine, you also need Default Application Credentials:

    gcloud auth application-default login

    This will open a web browser and prompt for a login to your Google account. On headless devices, you will need to create a service account. For more information, please see the authentication section.

  2. Install the berglas CLI using one of the following methods:

    • Install a pre-compiled binary for your operating system from the latest releases.

    • Use an official Docker container:

      docker run -it us-docker.pkg.dev/berglas/berglas/berglas

      Note: older Docker container images are available on Container Registry and Artifact Registry, but new versions are not published there.

    • Use Homebrew on macOS:

      brew install berglas

      Note: sometimes the Homebrew formula can be several versions behind.

    • Install from source (requires a working Go installation):

      go install github.com/GoogleCloudPlatform/berglas/v2@latest

  3. Export your project ID as an environment variable. The rest of this setup guide assumes this environment variable is set:

    export PROJECT_ID=my-gcp-project-id

    Please note, this is the project ID, not the project name or project number. You can find the project ID by running gcloud projects list or in the web UI.

Secret Manager Storage

  1. Enable required services on the project:

    gcloud services enable --project ${PROJECT_ID} \
  secretmanager.googleapis.com

Cloud Storage Storage

  1. Export your desired Cloud Storage bucket name. The rest of this setup guide assumes this environment variable is set:

    export BUCKET_ID=my-secrets

    Replace my-secrets with the name of your bucket. Set only the name, without the gs:// prefix. This bucket should not exist yet!

  2. Enable required services on the project:

    gcloud services enable --project ${PROJECT_ID} \
  cloudkms.googleapis.com \
  storage-api.googleapis.com \
  storage-component.googleapis.com

  3. Bootstrap a Berglas environment. This will create a new Cloud Storage bucket for storing secrets and a Cloud KMS key for encrypting data.

    berglas bootstrap --project $PROJECT_ID --bucket $BUCKET_ID

    This command uses the default values. You can customize the storage bucket and KMS key configuration using the optional flags. Run berglas bootstrap -h for more details.

    If you want full control over the creation of the Cloud Storage and Cloud KMS keys, please see the custom setup documentation.

  4. (Optional) Bootstrap a Berglas environment specifying a bucket location. By default the berglas bucket is created in the multi-regional location US. You can specify your location by using the following command. Please see the list of supported locations in the GCP bucket location documentation page

    export BUCKET_LOCATION=europe-west1
berglas bootstrap \
  --project $PROJECT_ID \
  --bucket $BUCKET_ID \
  --bucket-location $BUCKET_LOCATION

    This command uses the default values. You can customize the storage bucket and KMS key configuration using the optional flags. Run berglas bootstrap -h for more details.

    If you want full control over the creation of the Cloud Storage and Cloud KMS keys, please see the custom setup documentation.

  5. (Optional) Enable Cloud Audit logging on the bucket:

    Please note this will enable audit logging on all Cloud KMS keys and all Cloud Storage buckets in the project, which may incur additional costs.

    1. Download the exiting project IAM policy:

      gcloud projects get-iam-policy ${PROJECT_ID} > policy.yaml

    2. Add Cloud Audit logging for Cloud KMS and Cloud Storage:

      cat <<EOF >> policy.yaml
auditConfigs:
- auditLogConfigs:
  - logType: DATA_READ
  - logType: ADMIN_READ
  - logType: DATA_WRITE
  service: cloudkms.googleapis.com
- auditLogConfigs:
  - logType: ADMIN_READ
  - logType: DATA_READ
  - logType: DATA_WRITE
  service: storage.googleapis.com
EOF

    3. Submit the new policy:

      gcloud projects set-iam-policy ${PROJECT_ID} policy.yaml

    4. Remove the updated policy from local disk:

      rm policy.yaml

CLI Usage

  1. Create a secret:

    Using Secret Manager storage:

    berglas create sm://${PROJECT_ID}/foo my-secret-data

    Using Cloud Storage storage:

    berglas create ${BUCKET_ID}/foo my-secret-data \
  --key projects/${PROJECT_ID}/locations/global/keyRings/berglas/cryptoKeys/berglas-key

  2. Grant access to a secret:

    Using Secret Manager storage:

    berglas grant sm://${PROJECT_ID}/foo --member user:user@mydomain.com

    Using Cloud Storage storage:

    berglas grant ${BUCKET_ID}/foo --member user:user@mydomain.com

  3. Access a secret's data:

    Using Secret Manager storage:

    berglas access sm://${PROJECT_ID}/foo
my-secret-data

    Using Cloud Storage storage:

    berglas access ${BUCKET_ID}/foo
my-secret-data

  4. Spawn a child process with secrets populated in the child's environment:

    berglas exec -- myapp --flag-a --flag-b

    This will spawn myapp with an environment parsed by berglas.

  5. Access data from a specific version/generation of a secret:

    Using Secret Manager storage:

    berglas access sm://${PROJECT_ID}/foo#1
my-previous-secret-data

    Using Cloud Storage storage:

    berglas access ${BUCKET_ID}/foo#1563925940580201
my-previous-secret-data

  6. Revoke access to a secret:

    Using Secret Manager storage:

    berglas revoke sm://${PROJECT_ID}/foo --member user:user@mydomain.com
my-previous-secret-data

    Using Cloud Storage storage:

    berglas revoke ${BUCKET_ID}/foo --member user:user@mydomain.com

  7. Delete a secret:

    Using Secret Manager storage:

    berglas delete sm://${PROJECT_ID}/foo

    Using Cloud Storage storage:

    berglas delete ${BUCKET_ID}/foo

In addition to standard Unix exit codes, if the CLI exits with a known error, Berglas will exit with one of the following:

The only exception is berglas exec, which will exit with the exit status of its child command, if one was provided.

Integrations

Logging

Both the berglas CLI and berglas library support debug-style logging. This logging is off by default because it adds additional overhead and logs information that may be security-sensitive.

The default logging behavior for the berglas CLI is "text" (it can be changed with the --log-format flag). The default logging behavior for the berglas library is structured JSON which integrates well with Cloud Logging (it can be changed to any valid formatter and you can even inject your own logger).

Library Usage

Berglas is also a Go library that can be imported in Go projects:

import (
	_ "github.com/GoogleCloudPlatform/berglas/v2/pkg/auto"
)

When imported, the berglas package will:

  1. Download and decrypt any secrets that match the Berglas environment variable reference syntax in the environment.

  2. Replace the value for the environment variable with the decrypted secret.

You can also opt out of auto-parsing and call the library yourself instead:

import (
	"context"
	"log"
	"os"

	"github.com/GoogleCloudPlatform/berglas/v2/pkg/berglas"
)

func main() {
	ctx := context.Background()

	// This higher-level API parses the secret reference at the specified
	// environment variable, downloads and decrypts the secret, and replaces the
	// contents of the given environment variable with the secret result.
	if err := berglas.Replace(ctx, "MY_SECRET"); err != nil {
		log.Fatal(err)
	}

	// This lower-level API parses the secret reference, downloads and decrypts
	// the secret, and returns the result. This is useful if you need to mutate
	// the result.
	if v := os.Getenv("MY_SECRET"); v != "" {
		plaintext, err := berglas.Resolve(ctx, v)
		if err != nil {
			log.Fatal(err)
		}
		os.Unsetenv("MY_SECRET")
		os.Setenv("MY_OTHER_SECRET", string(plaintext))
	}
}

For more examples and documentation, please see the godoc.

Authentication

By default, Berglas uses Google Cloud Default Application Credentials. If you have gcloud installed locally, ensure you have application default credentials:

gcloud auth application-default login

On GCP services (like Cloud Build, Compute, etc), it will use the service account attached to the resource.

To use a specific service account, set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the filepath to the JSON file where your credentials reside on disk:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/credentials.json

To learn more, please see the Google Cloud Service Account documentation.

Authorization

To control who or what has access to a secret, use berglas grant and berglas revoke commands. These methods use Cloud IAM internally. Any service account or entity using Berglas will need to authorize using the cloud-platform scope.

Secret Manager Storage

Creating a secret requires roles/secretmanager.admin on Secret Manager in the project.

Accessing a secret requires roles/secretmanager.secretAccessor on the secret in Secret Manager.

Deleting a secret requires roles/secretmanager.admin on Secret Manager in the project.

Cloud Storage Storage

Creating a secret requires roles/storage.objectCreator on the Cloud Storage bucket and roles/cloudkms.cryptoKeyEncrypter on the Cloud KMS key.

Accessing a secret requires roles/storage.objectViewer on the Cloud Storage bucket and roles/cloudkms.cryptoKeyDecrypter on the Cloud KMS key.

Deleting a secret requires roles/storage.objectAdmin on the Cloud Storage bucket.

Implementation

Secret Manager Storage

This section describes the Secret Manager implementation. This knowledge is not required to use Berglas, but it is included for security-conscious/curious users who want to learn about how Berglas works internally to build a threat model.

  1. Berglas calls the Secret Manager API directly for all operations.

Cloud Storage Storage

This section describes the Cloud Storage implementation. This knowledge is not required to use Berglas, but it is included for security-conscious/curious users who want to learn about how Berglas works internally to build a threat model.

When encrypting a secret:

  1. Berglas generates an AES-256-GCM data encryption key (DEK) using Go's crypto package for each secret. (N.B. each secret has its own, unique DEK).

  2. Berglas encrypts the plaintext data using the locally-generated DEK, producing encrypted ciphertext, prepended with the AES-GCM nonce.

  3. Berglas encrypts the DEK using the specified Cloud KMS key, also known as a key encryption key (KEK). This process is called envelope encryption.

  4. Berglas stores the Cloud KMS key name, encrypted DEK, and encrypted ciphertext as a single blob in Cloud Storage.

When decrypting a secret:

  1. Berglas downloads the blob from Cloud Storage and separates the Cloud KMS key name, encrypted DEK, and ciphertext out of the blob.

  2. Berglas decrypts the DEK using Cloud KMS. This is part of envelope encryption.

  3. Berglas decrypts the ciphertext data locally using the decrypted DEK.

Security & Threat Model

See the security and threat model.

FAQ

Q: Should I use Berglas or Secret Manager? <br> Berglas is compatible with Secret Manager and offers convenience wrappers around managing secrets regardless of whether they reside in Cloud Storage or Secret Manager. New projects should investigate using Secret Manager directly as it has less operational overhead and complexity, but Berglas will continue to support Cloud Storage + Cloud KMS secrets.

Q: Is there a size limit on the data I can encrypt? <br> Berglas is targeted at application secrets like certificates, passwords, and API keys. While its possible to encrypt larger binary files like PDFs or images, Berglas uses a a GCM cipher mode to encrypt data, meaning the data must fit in memory and is limited to 64GiB.

Q: Why do you use envelope encryption instead of encrypting the data directly with Cloud KMS? <br> Envelope encryption allows for encrypting the data at the application layer, and it enables encryption of larger payloads, since Cloud KMS has a limit on the size of the payload it can encrypt. By using envelope encryption, Cloud KMS always encrypts a fixed size data (the AES-256-GCM key). This saves bandwidth (since large payloads are encrypted locally) and increases the size of the data which can be encrypted.

Q: Why does Berglas need permission to view my GCP resource? <br> Berglas communicates with the API to read the environment variables that were set on the resource at deploy time. Otherwise, a package could inject arbitrary environment variables in the Berglas format during application boot.

Q: I renamed a secret in Cloud Storage and now it fails to decrypt - why? <br> Berglas encrypts secrets with additional authenticated data including the name of the secret. This reduces the chance an attacker can escalate privilege by convincing someone to rename a secret so they can gain access.

Q: Why is it named Berglas? <br> Berglas is a famous magician who is best known for his secrets.

Contributing

Please see the contributing guidelines.

License

This library is licensed under Apache 2.0. Full license text is available in LICENSE.