You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Jian Zheng (Jira)" <ji...@apache.org> on 2020/09/21 15:01:00 UTC
[jira] [Created] (BEAM-10936) PubsubIO can't provide
PubsubClientFactory
Jian Zheng created BEAM-10936:
---------------------------------
Summary: PubsubIO can't provide PubsubClientFactory
Key: BEAM-10936
URL: https://issues.apache.org/jira/browse/BEAM-10936
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.23.0, 2.22.0, 2.21.0, 2.20.0, 2.19.0
Environment: Dataflow
Reporter: Jian Zheng
PubsubIO provides a method to pass the PubsubClientFactory.
{code:java}
//code
/**
* The default client to write to Pub/Sub is the {@link PubsubJsonClient}, created by the {@link
* PubsubJsonClient.PubsubJsonClientFactory}. This function allows to change the Pub/Sub client
* by providing another {@link PubsubClient.PubsubClientFactory} like the {@link
* PubsubGrpcClientFactory}.
*/
public Read<T> withClientFactory(PubsubClient.PubsubClientFactory factory) {
return toBuilder().setPubsubClientFactory(factory).build();
}
{code}
The comment here explains that can pass a PubsubClient.
However, the IncomingMessage used in the {color:#FF0000}pull(){color} method of PubsubClient's abstract method is actually an abstract internal class, in other words,
the abstract class PubsubClient can't be extended in external of the {color:#FF0000}org.apache.beam.sdk.io.gcp.pubsub{color} package.
To overcome the above problem, I created the org.apache.beam.sdk.io.gcp.pubsub directory to extend PubsubClient and passes it to the {color:#FF0000}PubsubIO.withClientFactory{color} method.
This seems to run locally without any problems, but when I submit my job to Dataflow, the withClientFactory method is useless and be replaced with the default PubsubJsonClientFactory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)