[jira] [Updated] (BEAM-10936) PubsubIO can't provide PubsubClientFactory

["Jian Zheng (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-10936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jian Zheng updated BEAM-10936:
------------------------------
    Description: 
PubsubIO provides a method to pass the PubsubClientFactory.
{code:java}
/**
 * The default client to write to Pub/Sub is the {@link PubsubJsonClient}, created by the {@link
 * PubsubJsonClient.PubsubJsonClientFactory}. This function allows to change the Pub/Sub client
 * by providing another {@link PubsubClient.PubsubClientFactory} like the {@link
 * PubsubGrpcClientFactory}.
 */
public Read<T> withClientFactory(PubsubClient.PubsubClientFactory factory) {
  return toBuilder().setPubsubClientFactory(factory).build();
}
{code}
The comment here explains that can pass a PubsubClient.

However, the IncomingMessage used in the {color:#ff0000}pull(){color} method of PubsubClient's abstract method is actually an abstract internal class, in other words, 
 the abstract class PubsubClient can't be extended in external of the {color:#ff0000}org.apache.beam.sdk.io.gcp.pubsub{color} package.

To overcome the above problem, I created the org.apache.beam.sdk.io.gcp.pubsub directory to extend PubsubClient and passes it to the {color:#ff0000}PubsubIO.withClientFactory{color} method.

This seems to run locally without any problems, but when I submit my job to Dataflow, the withClientFactory method is useless and be replaced with the default PubsubJsonClientFactory.

  was:
PubsubIO provides a method to pass the PubsubClientFactory.
{code:java}
//code
/**
 * The default client to write to Pub/Sub is the {@link PubsubJsonClient}, created by the {@link
 * PubsubJsonClient.PubsubJsonClientFactory}. This function allows to change the Pub/Sub client
 * by providing another {@link PubsubClient.PubsubClientFactory} like the {@link
 * PubsubGrpcClientFactory}.
 */
public Read<T> withClientFactory(PubsubClient.PubsubClientFactory factory) {
  return toBuilder().setPubsubClientFactory(factory).build();
}
{code}
The comment here explains that can pass a PubsubClient.

However, the IncomingMessage used in the {color:#FF0000}pull(){color} method of PubsubClient's abstract method is actually an abstract internal class, in other words, 
the abstract class PubsubClient can't be extended in external of the {color:#FF0000}org.apache.beam.sdk.io.gcp.pubsub{color} package.

To overcome the above problem, I created the org.apache.beam.sdk.io.gcp.pubsub directory to extend PubsubClient and passes it to the {color:#FF0000}PubsubIO.withClientFactory{color} method.

This seems to run locally without any problems, but when I submit my job to Dataflow, the withClientFactory method is useless and be replaced with the default PubsubJsonClientFactory.


> PubsubIO can't provide PubsubClientFactory  
> --------------------------------------------
>
>                 Key: BEAM-10936
>                 URL: https://issues.apache.org/jira/browse/BEAM-10936
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-gcp
>    Affects Versions: 2.19.0, 2.20.0, 2.21.0, 2.22.0, 2.23.0
>         Environment: Dataflow
>            Reporter: Jian Zheng
>            Priority: P2
>
> PubsubIO provides a method to pass the PubsubClientFactory.
> {code:java}
> /**
>  * The default client to write to Pub/Sub is the {@link PubsubJsonClient}, created by the {@link
>  * PubsubJsonClient.PubsubJsonClientFactory}. This function allows to change the Pub/Sub client
>  * by providing another {@link PubsubClient.PubsubClientFactory} like the {@link
>  * PubsubGrpcClientFactory}.
>  */
> public Read<T> withClientFactory(PubsubClient.PubsubClientFactory factory) {
>   return toBuilder().setPubsubClientFactory(factory).build();
> }
> {code}
> The comment here explains that can pass a PubsubClient.
> However, the IncomingMessage used in the {color:#ff0000}pull(){color} method of PubsubClient's abstract method is actually an abstract internal class, in other words, 
>  the abstract class PubsubClient can't be extended in external of the {color:#ff0000}org.apache.beam.sdk.io.gcp.pubsub{color} package.
> To overcome the above problem, I created the org.apache.beam.sdk.io.gcp.pubsub directory to extend PubsubClient and passes it to the {color:#ff0000}PubsubIO.withClientFactory{color} method.
> This seems to run locally without any problems, but when I submit my job to Dataflow, the withClientFactory method is useless and be replaced with the default PubsubJsonClientFactory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)