You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Logan HAUSPIE (JIRA)" <ji...@apache.org> on 2018/10/08 13:55:00 UTC
[jira] [Updated] (BEAM-5676) Allow to change the
PubsubClientFactory when using PubsubIO
[ https://issues.apache.org/jira/browse/BEAM-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Logan HAUSPIE updated BEAM-5676:
--------------------------------
Description:
When we use PubSub to push or pull messages, we currently have no choice of serialization implementation because PubsubIO use internaly the *_PubsubJsonClient.FACTORY_* _of type_ _*PubsubJsonClientFactory*_ _to serialize deserialize messages_.
It should be nice to be able to set a different factory (e.g. *_PubsubGrpcClientFactory_*) to decrease the size of messages (and then decrease the price of using Pubsub).
I guess It could be possible to write something like:
{{PubsubIO.Write<PubsubMessage> write =}}
{{ PubsubIO.writeMessages()}}
{{ .to("projects/project/topics/topic")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp")}}{{;}}
or
{{PubsubIO.Read<PubsubMessage> read = }}
{{ PubsubIO.readMessages()}}
{{ .fromSubscription("projects/project/subscriptions/name")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp");}}
I feel confident to do it if needed.
was:
When we use PubSub to push or pull messages, we currently have no choice of serialization implementation because PubsubIO use internaly the *_PubsubJsonClient.FACTORY_* _of type_ *__* *_PubsubJsonClientFactory_* _to serialize deserialize messages_.
It should be nice to be able to set a different factory (e.g. *_PubsubGrpcClientFactory_*) to decrease the size of messages (and then decrease the price of using Pubsub).
I guess It could be possible to write something like:
{{PubsubIO.Write<PubsubMessage> write =}}
{{ PubsubIO.writeMessages()}}
{{ .to("projects/project/topics/topic")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp")}}{{;}}
or
{{PubsubIO.Read<PubsubMessage> read = }}
{{ PubsubIO.readMessages()}}
{{ .fromSubscription("projects/project/subscriptions/name")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp");}}
I feel confident to do it if needed.
> Allow to change the PubsubClientFactory when using PubsubIO
> -----------------------------------------------------------
>
> Key: BEAM-5676
> URL: https://issues.apache.org/jira/browse/BEAM-5676
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Affects Versions: 2.7.0
> Reporter: Logan HAUSPIE
> Assignee: Chamikara Jayalath
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When we use PubSub to push or pull messages, we currently have no choice of serialization implementation because PubsubIO use internaly the *_PubsubJsonClient.FACTORY_* _of type_ _*PubsubJsonClientFactory*_ _to serialize deserialize messages_.
> It should be nice to be able to set a different factory (e.g. *_PubsubGrpcClientFactory_*) to decrease the size of messages (and then decrease the price of using Pubsub).
>
> I guess It could be possible to write something like:
> {{PubsubIO.Write<PubsubMessage> write =}}
> {{ PubsubIO.writeMessages()}}
> {{ .to("projects/project/topics/topic")}}
> {{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
> {{ .withTimestampAttribute("timestamp")}}{{;}}
> or
> {{PubsubIO.Read<PubsubMessage> read = }}
> {{ PubsubIO.readMessages()}}
> {{ .fromSubscription("projects/project/subscriptions/name")}}
> {{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
> {{ .withTimestampAttribute("timestamp");}}
>
> I feel confident to do it if needed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)