Home

Awesome

Copybara

A tool for transforming and moving code between repositories.

Copybara is a tool used internally at Google. It transforms and moves code between repositories.

Often, source code needs to exist in multiple repositories, and Copybara allows you to transform and move source code between these repositories. A common case is a project that involves maintaining a confidential repository and a public repository in sync.

Copybara requires you to choose one of the repositories to be the authoritative repository, so that there is always one source of truth. However, the tool allows contributions to any repository, and any repository can be used to cut a release.

The most common use case involves repetitive movement of code from one repository to another. Copybara can also be used for moving code once to a new repository.

Examples uses of Copybara include:

One of the main features of Copybara is that it is stateless, or more specifically, that it stores the state in the destination repository (As a label in the commit message). This allows several users (or a service) to use Copybara for the same config/repositories and get the same result.

Currently, the only supported type of repository is Git. Copybara is also able to read from Mercurial repositories, but the feature is still experimental. The extensible architecture allows adding bespoke origins and destinations for almost any use case. Official support for other repositories types will be added in the future.

Example

core.workflow(
    name = "default",
    origin = git.github_origin(
      url = "https://github.com/google/copybara.git",
      ref = "master",
    ),
    destination = git.destination(
        url = "file:///tmp/foo",
    ),

    # Copy everything but don't remove a README_INTERNAL.txt file if it exists.
    destination_files = glob(["third_party/copybara/**"], exclude = ["README_INTERNAL.txt"]),

    authoring = authoring.pass_thru("Default email <default@default.com>"),
    transformations = [
        core.replace(
                before = "//third_party/bazel/bashunit",
                after = "//another/path:bashunit",
                paths = glob(["**/BUILD"])),
        core.move("", "third_party/copybara")
    ],
)

Run:

$ (mkdir /tmp/foo ; cd /tmp/foo ; git init --bare)
$ copybara copy.bara.sky

Getting Started using Copybara

Copybara doesn't have a release process yet, so you need to compile from HEAD. In order to do that, you need to do the following:

System packages

These packages can be installed using the appropriate package manager for your system.

Arch Linux

Using Intellij with Bazel plugin

If you use Intellij and the Bazel plugin, use this project configuration:

directories:
  copybara/integration
  java/com/google/copybara
  javatests/com/google/copybara
  third_party

targets:
  //copybara/integration/...
  //java/com/google/copybara/...
  //javatests/com/google/copybara/...
  //third_party/...

Note: configuration files can be stored in any place, even in a local folder. We recommend using a VCS (like git) to store them; treat them as source code.

Building Copybara in an external Bazel workspace

There are convenience macros defined for all of Copybara's dependencies. Add the following code to your WORKSPACE file, replacing {{ sha256sum }} and {{ commit }} as necessary.

http_archive(
  name = "com_github_google_copybara",
  sha256 = "{{ sha256sum }}",
  strip_prefix = "copybara-{{ commit }}",
  url = "https://github.com/google/copybara/archive/{{ commit }}.zip",
)

load("@com_github_google_copybara//:repositories.bzl", "copybara_repositories")

copybara_repositories()

load("@com_github_google_copybara//:repositories.maven.bzl", "copybara_maven_repositories")

copybara_maven_repositories()

load("@com_github_google_copybara//:repositories.go.bzl", "copybara_go_repositories")

copybara_go_repositories()

You can then build and run the Copybara tool from within your workspace:

bazel run @com_github_google_copybara//java/com/google/copybara -- <args...>

Using Docker to build and run Copybara

NOTE: Docker use is currently experimental, and we encourage feedback or contributions.

You can build copybara using Docker like so

docker build --rm -t copybara .

Once this has finished building, you can run the image like so from the root of the code you are trying to use Copybara on:

docker run -it -v "$(pwd)":/usr/src/app copybara help

Environment variables

In addition to passing cmd args to the container, you can also set the following environment variables as an alternative:

docker run \
    -e COPYBARA_SUBCOMMAND='validate' \
    -e COPYBARA_CONFIG='other.config.sky' \
    -v "$(pwd)":/usr/src/app \
    -it copybara

Git Config and Credentials

There are a number of ways by which to share your git config and ssh credentials with the Docker container, an example is below:

docker run \
    -v ~/.gitconfig:/root/.gitconfig:ro \
    -v ~/.ssh:/root/.ssh \
    -v ${SSH_AUTH_SOCK}:${SSH_AUTH_SOCK} -e SSH_AUTH_SOCK
    -v "$(pwd)":/usr/src/app \
    -it copybara

Documentation

We are still working on the documentation. Here are some resources:

Contact us

If you have any questions about how Copybara works, please contact us at our mailing list.

Optional tips