You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Stephan Ewen (JIRA)" <ji...@apache.org> on 2018/05/07 19:32:00 UTC

[jira] [Commented] (FLINK-9311) PubSub connector

    [ https://issues.apache.org/jira/browse/FLINK-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466356#comment-16466356 ] 

Stephan Ewen commented on FLINK-9311:
-------------------------------------

The RichParallelSourceFunction is a good place to start.

What is the exactly-once model behind PubSub? Is it message ack/commit, like for example in RabbitMQ, or is it sequence-number/offset commits, like in Kafka/Kinesis?

> PubSub connector
> ----------------
>
>                 Key: FLINK-9311
>                 URL: https://issues.apache.org/jira/browse/FLINK-9311
>             Project: Flink
>          Issue Type: New Feature
>          Components: Streaming Connectors
>            Reporter: Richard Deurwaarder
>            Priority: Minor
>
> I would like start adding some google cloud connectors starting with a PubSub Source. I have a basic implementation ready but I want it to be able to:
>  * easily scale up (should I have it extend RichParallelSourceFunction for this?)
>  * Make it easier to provide the google cloud credentials. This would require being able to send some json string / ServiceAccount to the nodes when starting up this source.
> Could this be something that would be useful for others and added to the flink connectors repo?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)