Awesome

Google Cloud Storage Client for Java

Java idiomatic client for Cloud Storage.

• Product Documentation
• Client Library Documentation

Quickstart

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.49.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage</artifactId>
  </dependency>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage-control</artifactId>
  </dependency>
</dependencies>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-storage</artifactId>
  <version>2.44.1</version>
</dependency>
<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-storage-control</artifactId>
  <version>2.44.1</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.49.0')

implementation 'com.google.cloud:google-cloud-storage'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-storage:2.44.1'

If you are using SBT, add this to your dependencies:

libraryDependencies += "com.google.cloud" % "google-cloud-storage" % "2.44.1"

Authentication

See the Authentication section in the base directory's README.

Authorization

The client application making API calls must be granted authorization scopes required for the desired Cloud Storage APIs, and the authenticated principal must have the IAM role(s) required to access GCP resources using the Cloud Storage API calls.

Getting Started

Prerequisites

You will need a Google Cloud Platform Console project with the Cloud Storage API enabled. You will need to enable billing to use Google Cloud Storage. Follow these instructions to get your project set up. You will also need to set up the local development environment by installing the Google Cloud Command Line Interface and running the following commands in command line: gcloud auth login and gcloud config set project [YOUR PROJECT ID].

Installation and setup

You'll need to obtain the google-cloud-storage library. See the Quickstart section to add google-cloud-storage as a dependency in your code.

About Cloud Storage

Cloud Storage is a durable and highly available object storage service. Google Cloud Storage is almost infinitely scalable and guarantees consistency: when a write succeeds, the latest copy of the object will be returned to any GET, globally.

See the Cloud Storage client library docs to learn how to use this Cloud Storage Client Library.

About Storage Control

The Storage Control API lets you perform metadata-specific, control plane, and long-running operations.

The Storage Control API creates one space to perform metadata-specific, control plane, and long-running operations apart from the Storage API. Separating these operations from the Storage API improves API standardization and lets you run faster releases.

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>libraries-bom</artifactId>
            <version>26.44.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>google-cloud-storage-control</artifactId>
    </dependency>
</dependencies>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage-control</artifactId>
    <version>2.41.0</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.44.0')
implementation 'com.google.cloud:google-cloud-storage-control'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-storage-control:2.41.0'

Creating an authorized service object

To make authenticated requests to Google Cloud Storage, you must create a service object with credentials. You can then make API calls by calling methods on the Storage service object. The simplest way to authenticate is to use Application Default Credentials. These credentials are automatically inferred from your environment, so you only need the following code to create your service object:

import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

Storage storage = StorageOptions.getDefaultInstance().getService();

For other authentication options, see the Authentication page in Google Cloud Java.

Storing data

Stored objects are called "blobs" in google-cloud and are organized into containers called "buckets". Blob, a subclass of BlobInfo, adds a layer of service-related functionality over BlobInfo. Similarly, Bucket adds a layer of service-related functionality over BucketInfo. In this code snippet, we will create a new bucket and upload a blob to that bucket.

Add the following imports at the top of your file:

import static java.nio.charset.StandardCharsets.UTF_8;

import com.google.cloud.storage.Blob;
import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;

Then add the following code to create a bucket and upload a simple blob.

Important: Bucket names have to be globally unique (among all users of Cloud Storage). If you choose a bucket name that already exists, you'll get a helpful error message telling you to choose another name. In the code below, replace "my_unique_bucket" with a unique bucket name. See more about naming rules here.

// Create a bucket
String bucketName = "my_unique_bucket"; // Change this to something unique
Bucket bucket = storage.create(BucketInfo.of(bucketName));

// Upload a blob to the newly created bucket
BlobId blobId = BlobId.of(bucketName, "my_blob_name");
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
Blob blob = storage.create(blobInfo, "a simple blob".getBytes(UTF_8));

A complete example for creating a blob can be found at UploadObject.java.

At this point, you will be able to see your newly created bucket and blob on the Google Developers Console.

Retrieving data

Now that we have content uploaded to the server, we can see how to read data from the server. Add the following line to your program to get back the blob we uploaded.

BlobId blobId = BlobId.of(bucketName, "my_blob_name");
byte[] content = storage.readAllBytes(blobId);
String contentString = new String(content, UTF_8);

A complete example for accessing blobs can be found at DownloadObject.java.

Updating data

Another thing we may want to do is update a blob. The following snippet shows how to update a Storage blob if it exists.

BlobId blobId = BlobId.of(bucketName, "my_blob_name");
Blob blob = storage.get(blobId);
if (blob != null) {
  byte[] prevContent = blob.getContent();
  System.out.println(new String(prevContent, UTF_8));
  WritableByteChannel channel = blob.writer();
  channel.write(ByteBuffer.wrap("Updated content".getBytes(UTF_8)));
  channel.close();
}

Listing buckets and contents of buckets

Suppose that you've added more buckets and blobs, and now you want to see the names of your buckets and the contents of each one. Add the following code to list all your buckets and all the blobs inside each bucket.

// List all your buckets
System.out.println("My buckets:");
for (Bucket bucket : storage.list().iterateAll()) {
  System.out.println(bucket);

  // List all blobs in the bucket
  System.out.println("Blobs in the bucket:");
  for (Blob blob : bucket.list().iterateAll()) {
    System.out.println(blob);
  }
}

Complete source code

See ListObjects.java for a complete example.

Example Applications

• Bookshelf - An App Engine application that manages a virtual bookshelf.
  • This app uses google-cloud to interface with Cloud Datastore and Cloud Storage. It also uses Cloud SQL, another Google Cloud Platform service.
• Flexible Environment/Storage example - An app that uploads files to a public Cloud Storage bucket on the App Engine Flexible Environment runtime.

Samples

Samples are in the samples/ directory.

Sample	Source Code	Try it
Configure Retries	source code
Generate Signed Post Policy V4	source code
Get Service Account	source code
Quickstart Grpc Dp Sample	source code
Quickstart Grpc Sample	source code
Quickstart Sample	source code
Quickstart Storage Control Sample	source code
Add Bucket Default Owner	source code
Add Bucket Iam Conditional Binding	source code
Add Bucket Iam Member	source code
Add Bucket Label	source code
Add Bucket Owner	source code
Change Default Storage Class	source code
Configure Bucket Cors	source code
Create Bucket	source code
Create Bucket Dual Region	source code
Create Bucket Pub Sub Notification	source code
Create Bucket With Object Retention	source code
Create Bucket With Storage Class And Location	source code
Create Bucket With Turbo Replication	source code
Delete Bucket	source code
Delete Bucket Pub Sub Notification	source code
Disable Bucket Versioning	source code
Disable Default Event Based Hold	source code
Disable Lifecycle Management	source code
Disable Requester Pays	source code
Disable Uniform Bucket Level Access	source code
Enable Bucket Versioning	source code
Enable Default Event Based Hold	source code
Enable Lifecycle Management	source code
Enable Requester Pays	source code
Enable Uniform Bucket Level Access	source code
Get Bucket Autoclass	source code
Get Bucket Metadata	source code
Get Bucket Rpo	source code
Get Default Event Based Hold	source code
Get Public Access Prevention	source code
Get Requester Pays Status	source code
Get Retention Policy	source code
Get Uniform Bucket Level Access	source code
List Bucket Iam Members	source code
List Buckets	source code
List Pub Sub Notifications	source code
Lock Retention Policy	source code
Make Bucket Public	source code
Print Bucket Acl	source code
Print Bucket Acl Filter By User	source code
Print Pub Sub Notification	source code
Remove Bucket Cors	source code
Remove Bucket Default Kms Key	source code
Remove Bucket Default Owner	source code
Remove Bucket Iam Conditional Binding	source code
Remove Bucket Iam Member	source code
Remove Bucket Label	source code
Remove Bucket Owner	source code
Remove Retention Policy	source code
Set Async Turbo Rpo	source code
Set Bucket Autoclass	source code
Set Bucket Default Kms Key	source code
Set Bucket Website Info	source code
Set Client Endpoint	source code
Set Default Rpo	source code
Set Public Access Prevention Enforced	source code
Set Public Access Prevention Inherited	source code
Set Retention Policy	source code
Create Folder	source code
Create Hierarchical Namespace Bucket	source code
Delete Folder	source code
Get Folder	source code
List Folders	source code
Rename Folder	source code
Activate Hmac Key	source code
Create Hmac Key	source code
Deactivate Hmac Key	source code
Delete Hmac Key	source code
Get Hmac Key	source code
List Hmac Keys	source code
Create Managed Folder	source code
Delete Managed Folder	source code
Get Managed Folder	source code
List Managed Folders	source code
Add File Owner	source code
Batch Set Object Metadata	source code
Change Object Csek To Kms	source code
Change Object Storage Class	source code
Compose Object	source code
Copy Object	source code
Copy Old Version Of Object	source code
Delete Object	source code
Delete Old Version Of Object	source code
Download Byte Range	source code
Download Encrypted Object	source code
Download Object	source code
Download Object Into Memory	source code
Download Public Object	source code
Download Requester Pays Object	source code
Generate Encryption Key	source code
Generate V4 Get Object Signed Url	source code
Generate V4 Put Object Signed Url	source code
Get Object Metadata	source code
List Objects	source code
List Objects With Old Versions	source code
List Objects With Prefix	source code
Make Object Public	source code
Move Object	source code
Print File Acl	source code
Print File Acl For User	source code
Release Event Based Hold	source code
Release Temporary Hold	source code
Remove File Owner	source code
Rotate Object Encryption Key	source code
Set Event Based Hold	source code
Set Object Metadata	source code
Set Object Retention Policy	source code
Set Temporary Hold	source code
Stream Object Download	source code
Stream Object Upload	source code
Upload Encrypted Object	source code
Upload Kms Encrypted Object	source code
Upload Object	source code
Upload Object From Memory	source code
Allow Divide And Conquer Download	source code
Allow Parallel Composite Upload	source code
Download Bucket	source code
Download Many	source code
Upload Directory	source code
Upload Many	source code

Troubleshooting

To get help, follow the instructions in the shared Troubleshooting document.

Transport

Cloud Storage uses HTTP/JSON for the transport layer.

Supported Java Versions

Java 8 or above is required for using this client.

Google's Java client libraries, Google Cloud Client Libraries and Google Cloud API Libraries, follow the Oracle Java SE support roadmap (see the Oracle Java SE Product Releases section).

For new development

In general, new feature development occurs with support for the lowest Java LTS version covered by Oracle's Premier Support (which typically lasts 5 years from initial General Availability). If the minimum required JVM for a given library is changed, it is accompanied by a semver major release.

Java 11 and (in September 2021) Java 17 are the best choices for new development.

Keeping production systems current

Google tests its client libraries with all current LTS versions covered by Oracle's Extended Support (which typically lasts 8 years from initial General Availability).

Legacy support

Google's client libraries support legacy versions of Java runtimes with long term stable libraries that don't receive feature updates on a best efforts basis as it may not be possible to backport all patches.

Google provides updates on a best efforts basis to apps that continue to use Java 7, though apps might need to upgrade to current versions of the library that supports their JVM.

Where to find specific information

The latest versions and the supported Java versions are identified on the individual GitHub repository github.com/GoogleAPIs/java-SERVICENAME and on google-cloud-java.

Versioning

This library follows Semantic Versioning, but does update Storage interface to introduce new methods which can break your implementations if you implement this interface for testing purposes.

Contributing

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING for more information how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

License

Apache 2.0 - See LICENSE for more information.

CI Status

Java Version	Status
Java 8
Java 8 OSX
Java 8 Windows
Java 11

Java is a registered trademark of Oracle and/or its affiliates.