You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Steve Niemitz (Jira)" <ji...@apache.org> on 2021/04/08 04:42:00 UTC
[jira] [Commented] (BEAM-10670) Make non-portable Splittable DoFn
the only option when executing Java "Read" transforms
[ https://issues.apache.org/jira/browse/BEAM-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316858#comment-17316858 ]
Steve Niemitz commented on BEAM-10670:
--------------------------------------
This ticket seems possibly out of date now, but I did want to come in and leave a comment that, until something is fixed with PubsubIO on the DirectRunner (and I'd assume any other runner other than dataflow...), using use_deprecated_reads is the only reasonable option for running pipelines that use it, otherwise they'll go for long periods of time without emitting any elements.
> Make non-portable Splittable DoFn the only option when executing Java "Read" transforms
> ---------------------------------------------------------------------------------------
>
> Key: BEAM-10670
> URL: https://issues.apache.org/jira/browse/BEAM-10670
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Luke Cwik
> Priority: P3
> Labels: Clarified
> Time Spent: 37h 50m
> Remaining Estimate: 0h
>
> All runners seem to be capable of migrating to splittable DoFn for non-portable execution except for Dataflow runner v1 which will internalize the current primitive read implementation that is shared across runner implementations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)