Skip to content

googleapis/java-pubsub

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Google Cloud Pub/Sub Client for Java

Java idiomatic client for Cloud Pub/Sub.

Maven Stability

Quickstart

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.40.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-pubsub</artifactId>
  </dependency>

</dependencies>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-pubsub</artifactId>
  <version>1.129.6</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.40.0')

implementation 'com.google.cloud:google-cloud-pubsub'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-pubsub:1.129.6'

If you are using SBT, add this to your dependencies:

libraryDependencies  = "com.google.cloud" % "google-cloud-pubsub" % "1.129.6"

Authentication

See the Authentication section in the base directory's README.

Authorization

The client application making API calls must be granted authorization scopes required for the desired Cloud Pub/Sub APIs, and the authenticated principal must have the IAM role(s) required to access GCP resources using the Cloud Pub/Sub API calls.

Getting Started

Prerequisites

You will need a Google Cloud Platform Console project with the Cloud Pub/Sub API enabled. You will need to enable billing to use Google Cloud Pub/Sub. Follow these instructions to get your project set up. You will also need to set up the local development environment by installing the Google Cloud Command Line Interface and running the following commands in command line: gcloud auth login and gcloud config set project [YOUR PROJECT ID].

Installation and setup

You'll need to obtain the google-cloud-pubsub library. See the Quickstart section to add google-cloud-pubsub as a dependency in your code.

About Cloud Pub/Sub

Cloud Pub/Sub is designed to provide reliable, many-to-many, asynchronous messaging between applications. Publisher applications can send messages to a topic and other applications can subscribe to that topic to receive the messages. By decoupling senders and receivers, Google Cloud Pub/Sub allows developers to communicate between independently written applications.

See the Cloud Pub/Sub client library docs to learn how to use this Cloud Pub/Sub Client Library.

Creating a topic

With Pub/Sub you can create topics. A topic is a named resource to which messages are sent by publishers. Add the following imports at the top of your file:

import com.google.cloud.pubsub.v1.TopicAdminClient;
import com.google.pubsub.v1.TopicName;

Then, to create the topic, use the following code:

TopicName topic = TopicName.of("test-project", "test-topic");
try (TopicAdminClient topicAdminClient = TopicAdminClient.create()) {
  topicAdminClient.createTopic(topic);
}

Publishing messages

With Pub/Sub you can publish messages to a topic. Add the following import at the top of your file:

import com.google.api.core.ApiFuture;
import com.google.api.core.ApiFutureCallback;
import com.google.api.core.ApiFutures;
import com.google.cloud.pubsub.v1.Publisher;
import com.google.common.util.concurrent.MoreExecutors;
import com.google.protobuf.ByteString;
import com.google.pubsub.v1.PubsubMessage;

Then, to publish messages asynchronously, use the following code:

Publisher publisher = null;
try {
  publisher = Publisher.newBuilder(topic).build();
  ByteString data = ByteString.copyFromUtf8("my-message");
  PubsubMessage pubsubMessage = PubsubMessage.newBuilder().setData(data).build();
  ApiFuture<String> messageIdFuture = publisher.publish(pubsubMessage);
  ApiFutures.addCallback(messageIdFuture, new ApiFutureCallback<String>() {
    public void onSuccess(String messageId) {
      System.out.println("published with message id: "   messageId);
    }

    public void onFailure(Throwable t) {
      System.out.println("failed to publish: "   t);
    }
  }, MoreExecutors.directExecutor());
  //...
} finally {
  if (publisher != null) {
    publisher.shutdown();
    publisher.awaitTermination(1, TimeUnit.MINUTES);
  }
}

Creating a subscription

With Pub/Sub you can create subscriptions. A subscription represents the stream of messages from a single, specific topic. Add the following imports at the top of your file:

import com.google.cloud.pubsub.v1.SubscriptionAdminClient;
import com.google.pubsub.v1.PushConfig;
import com.google.pubsub.v1.SubscriptionName;
import com.google.pubsub.v1.TopicName;

Then, to create the subscription, use the following code:

TopicName topic = TopicName.of("test-project", "test-topic");
SubscriptionName subscription = SubscriptionName.of("test-project", "test-subscription");

try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {
  subscriptionAdminClient.createSubscription(subscription, topic, PushConfig.getDefaultInstance(), 0);
}

Pulling messages

With Pub/Sub you can pull messages from a subscription. Add the following imports at the top of your file:

import com.google.cloud.pubsub.v1.AckReplyConsumer;
import com.google.cloud.pubsub.v1.MessageReceiver;
import com.google.cloud.pubsub.v1.Subscriber;
import com.google.common.util.concurrent.MoreExecutors;
import com.google.pubsub.v1.PubsubMessage;
import com.google.pubsub.v1.SubscriptionName;
import com.google.pubsub.v1.TopicName;

Then, to pull messages asynchronously, use the following code:

SubscriptionName subscription = SubscriptionName.of("test-project", "test-subscription");

MessageReceiver receiver =
  new MessageReceiver() {
    @Override
    public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
      System.out.println("got message: "   message.getData().toStringUtf8());
      consumer.ack();
    }
  };

Subscriber subscriber = null;
try {
  subscriber = Subscriber.newBuilder(subscription.toString(), receiver).build();
  subscriber.addListener(
    new Subscriber.Listener() {
      @Override
      public void failed(Subscriber.State from, Throwable failure) {
        // Handle failure. This is called when the Subscriber encountered a fatal error and is shutting down.
        System.err.println(failure);
      }
    },
    MoreExecutors.directExecutor());
  subscriber.startAsync().awaitRunning();
  //...
} finally {
  if (subscriber != null) {
    subscriber.stopAsync();
  }
}

Complete source code

In CreateTopicAndPublishMessages.java and CreateSubscriptionAndConsumeMessages.java we put together all the code shown above into two programs. The programs assume that you are running on Compute Engine, App Engine Flexible or from your own desktop.

Samples

Samples are in the samples/ directory.

Sample Source Code Try it
Native Image Pub Sub Sample source code Open in Cloud Shell
Publish Operations source code Open in Cloud Shell
Commit Avro Schema Example source code Open in Cloud Shell
Commit Proto Schema Example source code Open in Cloud Shell
Create Avro Schema Example source code Open in Cloud Shell
Create Big Query Subscription Example source code Open in Cloud Shell
Create Cloud Storage Subscription Example source code Open in Cloud Shell
Create Proto Schema Example source code Open in Cloud Shell
Create Pull Subscription Example source code Open in Cloud Shell
Create Push Subscription Example source code Open in Cloud Shell
Create Subscription With Dead Letter Policy Example source code Open in Cloud Shell
Create Subscription With Exactly Once Delivery source code Open in Cloud Shell
Create Subscription With Filtering source code Open in Cloud Shell
Create Subscription With Ordering source code Open in Cloud Shell
Create Topic Example source code Open in Cloud Shell
Create Topic With Kinesis Ingestion Example source code Open in Cloud Shell
Create Topic With Schema Example source code Open in Cloud Shell
Create Topic With Schema Revisions Example source code Open in Cloud Shell
Create Unwrapped Push Subscription Example source code Open in Cloud Shell
Delete Schema Example source code Open in Cloud Shell
Delete Schema Revision Example source code Open in Cloud Shell
Delete Subscription Example source code Open in Cloud Shell
Delete Topic Example source code Open in Cloud Shell
Detach Subscription Example source code Open in Cloud Shell
Get Schema Example source code Open in Cloud Shell
Get Schema Revision Example source code Open in Cloud Shell
Get Subscription Policy Example source code Open in Cloud Shell
Get Topic Policy Example source code Open in Cloud Shell
List Schema Revisions Example source code Open in Cloud Shell
List Schemas Example source code Open in Cloud Shell
List Subscriptions In Project Example source code Open in Cloud Shell
List Subscriptions In Topic Example source code Open in Cloud Shell
List Topics Example source code Open in Cloud Shell
Publish Avro Records Example source code Open in Cloud Shell
Publish Protobuf Messages Example source code Open in Cloud Shell
Publish With Batch Settings Example source code Open in Cloud Shell
Publish With Concurrency Control Example source code Open in Cloud Shell
Publish With Custom Attributes Example source code Open in Cloud Shell
Publish With Error Handler Example source code Open in Cloud Shell
Publish With Flow Control Example source code Open in Cloud Shell
Publish With Grpc Compression Example source code Open in Cloud Shell
Publish With Ordering Keys source code Open in Cloud Shell
Publish With Retry Settings Example source code Open in Cloud Shell
Publisher Example source code Open in Cloud Shell
Receive Messages With Delivery Attempts Example source code Open in Cloud Shell
Remove Dead Letter Policy Example source code Open in Cloud Shell
Resume Publish With Ordering Keys source code Open in Cloud Shell
Rollback Schema Example source code Open in Cloud Shell
Set Subscription Policy Example source code Open in Cloud Shell
Set Topic Policy Example source code Open in Cloud Shell
Subscribe Async Example source code Open in Cloud Shell
Subscribe Sync Example source code Open in Cloud Shell
Subscribe Sync With Lease Example source code Open in Cloud Shell
Subscribe With Avro Schema Example source code Open in Cloud Shell
Subscribe With Avro Schema Revisions Example source code Open in Cloud Shell
Subscribe With Concurrency Control Example source code Open in Cloud Shell
Subscribe With Custom Attributes Example source code Open in Cloud Shell
Subscribe With Error Listener Example source code Open in Cloud Shell
Subscribe With Exactly Once Consumer With Response Example source code Open in Cloud Shell
Subscribe With Flow Control Settings Example source code Open in Cloud Shell
Subscribe With Proto Schema Example source code Open in Cloud Shell
Test Subscription Permissions Example source code Open in Cloud Shell
Test Topic Permissions Example source code Open in Cloud Shell
Update Dead Letter Policy Example source code Open in Cloud Shell
Update Push Configuration Example source code Open in Cloud Shell
Update Topic Schema Example source code Open in Cloud Shell
Update Topic Type Example source code Open in Cloud Shell
Use Pub Sub Emulator Example source code Open in Cloud Shell
State source code Open in Cloud Shell
State Proto source code Open in Cloud Shell

Troubleshooting

To get help, follow the instructions in the shared Troubleshooting document.

Supported Java Versions

Java 8 or above is required for using this client.

Google's Java client libraries, Google Cloud Client Libraries and Google Cloud API Libraries, follow the Oracle Java SE support roadmap (see the Oracle Java SE Product Releases section).

For new development

In general, new feature development occurs with support for the lowest Java LTS version covered by Oracle's Premier Support (which typically lasts 5 years from initial General Availability). If the minimum required JVM for a given library is changed, it is accompanied by a semver major release.

Java 11 and (in September 2021) Java 17 are the best choices for new development.

Keeping production systems current

Google tests its client libraries with all current LTS versions covered by Oracle's Extended Support (which typically lasts 8 years from initial General Availability).

Legacy support

Google's client libraries support legacy versions of Java runtimes with long term stable libraries that don't receive feature updates on a best efforts basis as it may not be possible to backport all patches.

Google provides updates on a best efforts basis to apps that continue to use Java 7, though apps might need to upgrade to current versions of the library that supports their JVM.

Where to find specific information

The latest versions and the supported Java versions are identified on the individual GitHub repository github.com/GoogleAPIs/java-SERVICENAME and on google-cloud-java.

Versioning

This library follows Semantic Versioning.

Contributing

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING for more information how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

License

Apache 2.0 - See LICENSE for more information.

CI Status

Java Version Status
Java 8 Kokoro CI
Java 8 OSX Kokoro CI
Java 8 Windows Kokoro CI
Java 11 Kokoro CI

Java is a registered trademark of Oracle and/or its affiliates.