[jira] [Updated] (BEAM-12455) Have a --wait_until_finish=[True|False] pipeline option

["Valentyn Tymofieiev (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Valentyn Tymofieiev updated BEAM-12455:
---------------------------------------
    Status: Open  (was: Triage Needed)

> Have a --wait_until_finish=[True|False] pipeline option
> -------------------------------------------------------
>
>                 Key: BEAM-12455
>                 URL: https://issues.apache.org/jira/browse/BEAM-12455
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-py-core
>            Reporter: David Cavazos
>            Priority: P3
>
> Currently, if we use the `with` statement to run a pipeline, it waits until finish by default with no way to configure it. The only way to run without waiting is by calling `pipeline.run()` explicitly without the `with` statement context. This is not consistent and users now have to be aware of two different ways of calling.
> Ideally, we would like to define pipelines with the `with` statement always to have a "single way to do things". Running without waiting is useful, for example, to run a pipeline in Cloud Build, Cloud Run, templates, or other serverless frameworks.
> Right now the only way to do it is by changing the code to remove the `with` statement and calling `pipeline.run()` manually, which might not be always possible. With a pipeline option we can just pass that from the command line arguments, or even when testing.
> I'm opening this for Python since many Python samples would greatly benefit from this, but I can see this being useful in Java and Go as well.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)