You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "COURAGE NOKO (JIRA)" <ji...@apache.org> on 2019/04/10 12:24:00 UTC
[jira] [Commented] (BEAM-2715) Expose PubsubSource to create
UnboundedSource and utilize withMaxNumRecords from
BoundedReadFromUnboundedSource
[ https://issues.apache.org/jira/browse/BEAM-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814407#comment-16814407 ]
COURAGE NOKO commented on BEAM-2715:
------------------------------------
[~kenn] Yes, this is still an outstanding request. We found this to be useful for end-to-end tests for streaming pipelines.
> Expose PubsubSource to create UnboundedSource and utilize withMaxNumRecords from BoundedReadFromUnboundedSource
> ---------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-2715
> URL: https://issues.apache.org/jira/browse/BEAM-2715
> Project: Beam
> Issue Type: New Feature
> Components: io-java-gcp
> Reporter: Adam Levy
> Priority: Major
> Labels: pubsub, triaged
>
> We are ingesting from a Pubsub Read and are attempting to mimic the maxNumRecords that was available in 0.6.0. In order to do this we would need to utilize withMaxNumRecords from the BoundedReadFromUnboundedSource class. We would need to utilize the PubsubSource class to create the UnboundedSource from Pubsub. Would it be possible to expose PubsubSource? Currently what is the recommended way to create a bounded read from Pubsub with a set number of records?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)