[jira] [Comment Edited] (BEAM-106) Native support for conditional iteration

["Luke Cwik (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000494#comment-17000494 ] 

Luke Cwik edited comment on BEAM-106 at 12/20/19 12:04 AM:
-----------------------------------------------------------

[~nishant009]:

Does using a glob expression for the path work when specifying a [file based read transform|https://beam.apache.org/releases/pydoc/2.2.0/apache_beam.io.filebasedsource.html] (e.g. gs://bucket/a/b/c/** for the file_pattern)?

 

 


was (Author: lcwik):
[~nishant009]:

Does using a glob expression for the path work when specifying a [file based read transform|https://beam.apache.org/releases/pydoc/2.2.0/apache_beam.io.filebasedsource.html] (e.g. gs://bucket/a/b/c/**)?

 

 

> Native support for conditional iteration
> ----------------------------------------
>
>                 Key: BEAM-106
>                 URL: https://issues.apache.org/jira/browse/BEAM-106
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-ideas
>            Reporter: Luke Cwik
>            Priority: Major
>
> Ported from: https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/50
> There are a variety of use cases which would benefit from native support for conditional iteration.
> For instance, http://stackoverflow.com/questions/31654421/conditional-iterations-in-google-cloud-dataflow/31659923?noredirect=1#comment51264604_31659923 asks about being able to write a loop like the following:
> {code}
> PCollection data  = ...
> while(needsMoreWork(data)) {
>   data = doAStep(data)
> }
> {code}
> If there are specific use cases please let us know the details. In the future we will use this issue to post progress updates.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)