You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "damccorm (via GitHub)" <gi...@apache.org> on 2024/04/08 20:57:27 UTC
[PR] Revert "Revert #30533: Automatically execute unbounded pipelines in streaming mode." [beam]
damccorm opened a new pull request, #30894:
URL: https://github.com/apache/beam/pull/30894
Reverts apache/beam#30706
This change was initially correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Revert "Revert #30533: Automatically execute unbounded pipelines in streaming mode." [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #30894:
URL: https://github.com/apache/beam/pull/30894#issuecomment-2052510228
We're taking a different (runner driven) approach here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Revert "Revert #30533: Automatically execute unbounded pipelines in streaming mode." [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #30894:
URL: https://github.com/apache/beam/pull/30894#discussion_r1556620836
##########
sdks/python/apache_beam/runners/dataflow/dataflow_runner.py:
##########
@@ -415,6 +416,12 @@ def run_pipeline(self, pipeline, options, pipeline_proto=None):
self.proto_pipeline, self.proto_context = pipeline.to_runner_api(
return_context=True, default_environment=self._default_environment)
+ if any(pcoll.is_bounded == beam_runner_api_pb2.IsBounded.UNBOUNDED
+ for pcoll in self.proto_pipeline.components.pcollections.values()):
+ options.view_as(StandardOptions).streaming = True
Review Comment:
if users specify streaming, can we honor the one specified by the users? This at least can let users keep the current behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Revert "Revert #30533: Automatically execute unbounded pipelines in streaming mode." [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed pull request #30894: Revert "Revert #30533: Automatically execute unbounded pipelines in streaming mode."
URL: https://github.com/apache/beam/pull/30894
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org