You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "damccorm (via GitHub)" <gi...@apache.org> on 2025/04/11 19:52:14 UTC
[PR] Enable prism by default (where supported) [beam]
damccorm opened a new pull request, #34612:
URL: https://github.com/apache/beam/pull/34612
Changes the default Python runner to be prism (when prism is supported). Excludes it from interactive environments for now.
TODO: Wait for next release cut so this has time to bake. Then add entry to CHANGES.md before submitting
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205030567
##########
sdks/python/apache_beam/io/fileio_test.py:
##########
@@ -106,7 +106,7 @@ def test_match_files_one_directory_failure1(self):
files.append(self._create_temp_file(dir=directories[0]))
files.append(self._create_temp_file(dir=directories[0]))
- with self.assertRaises(beam.io.filesystem.BeamIOError):
+ with self.assertRaises(Exception):
Review Comment:
When a Runtime Exception is raised in prism, can we use `[assertRaisesRegex](https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertRaisesRegex)` instead to match exception message?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-2997595530
Current persistent (or possibly persistent) test failures:
**[beam_PreCommit_Python_ML](https://github.com/apache/beam/actions/runs/15831111506/job/44623727892?pr=34612#logs)** and **[Python Unit Tests](https://github.com/apache/beam/actions/runs/15831111847/job/44623728997?pr=34612#logs)**:
- `apache_beam.ml.ts.util_test.PeriodicStreamTest` - a few failures, but this is a bad test. See https://github.com/apache/beam/pull/35300#issuecomment-2997517699 - if needed can fix here, but I'd rather we make some changes upstream
**[beam_PreCommit_Python](https://github.com/apache/beam/actions/runs/15831111491/job/44623727865?pr=34612#logs)**:
- apache_beam/io/textio_test.py::TextSinkTest::test_write_pipeline_footer - not sure what the issue is yet
**[beam_PreCommit_Python_Transforms](https://github.com/apache/beam/actions/runs/15831111519/job/44623727853?pr=34612#logs)**:
- apache_beam/transforms/userstate_test.py::StatefulDoFnOnDirectRunnerTest::test_stateful_set_state_clean_portably
**Linting/formatting** - I'll deal with this later
There are also some other flakes, which may or may not be related to this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2098351811
##########
sdks/python/apache_beam/yaml/yaml_mapping_test.py:
##########
@@ -175,7 +175,7 @@ def test_validate(self):
label='Errors')
def test_validate_explicit_types(self):
- with self.assertRaisesRegex(TypeError, r'.*violates schema.*'):
+ with self.assertRaisesRegex(RuntimeError, r'TypeError.*violates schema.*'):
Review Comment:
How about asserting just Exception rather than that it must now be a RuntimeError? (For all of these.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205100528
##########
sdks/python/apache_beam/transforms/util_test.py:
##########
@@ -254,8 +254,8 @@ def test_constant_batch_no_metrics(self):
self.assertEqual(len(results["distributions"]), 0)
def test_grows_to_max_batch(self):
- # Assumes a single bundle...
- with TestPipeline() as p:
+ # Assumes a single bundle, so we pin to the FnApiRunner
Review Comment:
Do we have a reliable way of getting elements in a single bundle? Maybe only for testing purpose. This seems to be a recurrent issue I meet when I am working on some other tests.
Alternatively, we may be able to change the tests so that they won't rely on this assumption, but it is not working all the time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-3073632744
There were failing postcommit tests, but not on prism
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-3079569983
> There are tests failing after this PR, could it be related?
>
> https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Xlang_Gcp_Direct.yml?query=event%3Aschedule
>
> https://github.com/apache/beam/actions/workflows/beam_PostCommit_Yaml_Xlang_Direct.yml?query=event%3Aschedule
I think it likely is - https://github.com/apache/beam/pull/35608 tested these workflows with extended timeouts and they still failed. Temporarily reverting the core of this change here - https://github.com/apache/beam/pull/35610
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2114449972
##########
sdks/python/apache_beam/yaml/yaml_mapping_test.py:
##########
@@ -175,7 +175,7 @@ def test_validate(self):
label='Errors')
def test_validate_explicit_types(self):
- with self.assertRaisesRegex(TypeError, r'.*violates schema.*'):
+ with self.assertRaisesRegex(RuntimeError, r'TypeError.*violates schema.*'):
Review Comment:
Sure, I can get behind that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205077410
##########
sdks/python/apache_beam/runners/direct/direct_runner.py:
##########
@@ -136,41 +176,40 @@ def visit_transform(self, applied_ptransform):
if userstate.is_stateful_dofn(dofn):
# https://github.com/apache/beam/issues/32786 -
# Remove once Real time clock is used.
- _, timer_specs = userstate.get_dofn_specs(dofn)
+ state_specs, timer_specs = userstate.get_dofn_specs(dofn)
for timer in timer_specs:
if timer.time_domain == TimeDomain.REAL_TIME:
self.supported_by_prism_runner = False
- tryingPrism = False
+ for state in state_specs:
+ if isinstance(state, userstate.CombiningValueStateSpec):
+ self.supported_by_prism_runner = False
+ # TODO(https://github.com/apache/beam/issues/33623): Prism seems to
Review Comment:
Interesting, I recently fixed a bug when prism handles sliding window (#35385). Just want to reference here in case it is related.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205071338
##########
sdks/python/apache_beam/runners/direct/direct_runner.py:
##########
@@ -112,9 +118,25 @@ def visit_transform(self, applied_ptransform):
class _PrismRunnerSupportVisitor(PipelineVisitor):
"""Visitor determining if a Pipeline can be run on the PrismRunner."""
- def accept(self, pipeline):
+ def accept(self, pipeline, is_interactive):
+ all_options = options.get_all_options()
self.supported_by_prism_runner = True
- pipeline.visit(self)
+ # TODO(https://github.com/apache/beam/issues/33623): Prism currently
+ # double fires on AfterCount trigger, once appropriately, and once
+ # incorrectly at the end of the window. This if condition could be
+ # more targeted, but for now we'll just ignore all unsafe triggers.
+ if pipeline.allow_unsafe_triggers:
+ self.supported_by_prism_runner = False
+ # TODO(https://github.com/apache/beam/issues/33623): Prism currently
Review Comment:
What else problems do we have here for prism to run in interactive mode?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-2873379943
I currently suspect that https://github.com/apache/beam/blob/0c92ee9343dfeadc1d2f3ac3d51362000464cda5/sdks/python/apache_beam/transforms/ptransform.py#L579 is the source of a lot of the test failures and that we're not handling applying transforms to non-PCollections in a Prism-friendly way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-3024752014
R: @shunping
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2093328464
##########
sdks/python/apache_beam/examples/complete/game/leader_board_test.py:
##########
@@ -65,7 +65,7 @@ def test_leader_board_users(self):
result = (
self.create_data(p)
| leader_board.CalculateUserScores(allowed_lateness=120))
- assert_that(result, equal_to([]))
+ assert_that(result, equal_to([('user1_team1', 50), ('user2_team2', 2), ('user3_team3', 8), ('user4_team3', 5)]))
Review Comment:
This should not be empty - Prism is handling this correctly, the FnApiRunner was dropping data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205030567
##########
sdks/python/apache_beam/io/fileio_test.py:
##########
@@ -106,7 +106,7 @@ def test_match_files_one_directory_failure1(self):
files.append(self._create_temp_file(dir=directories[0]))
files.append(self._create_temp_file(dir=directories[0]))
- with self.assertRaises(beam.io.filesystem.BeamIOError):
+ with self.assertRaises(Exception):
Review Comment:
When a Runtime Exception is raised in prism, maybe we can improve this to use [assertRaisesRegex](https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertRaisesRegex) instead to match exception message? Just like some example in `sdks/python/apache_beam/ml/inference/pytorch_inference_test.py` below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205100528
##########
sdks/python/apache_beam/transforms/util_test.py:
##########
@@ -254,8 +254,8 @@ def test_constant_batch_no_metrics(self):
self.assertEqual(len(results["distributions"]), 0)
def test_grows_to_max_batch(self):
- # Assumes a single bundle...
- with TestPipeline() as p:
+ # Assumes a single bundle, so we pin to the FnApiRunner
Review Comment:
Do have a reliable way of getting elements in a single bundle? Maybe only for testing purpose. This seems to be a recurrent issue I met when I am working on some other tests.
Alternatively, we may be able to change the tests so that they won't rely on this assumption, but it is not working all the time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2093328464
##########
sdks/python/apache_beam/examples/complete/game/leader_board_test.py:
##########
@@ -65,7 +65,7 @@ def test_leader_board_users(self):
result = (
self.create_data(p)
| leader_board.CalculateUserScores(allowed_lateness=120))
- assert_that(result, equal_to([]))
+ assert_that(result, equal_to([('user1_team1', 50), ('user2_team2', 2), ('user3_team3', 8), ('user4_team3', 5)]))
Review Comment:
This should not be empty - Prism is handling this correctly, the FnApiRunner was dropping data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2124795100
##########
sdks/python/apache_beam/runners/portability/prism_runner.py:
##########
@@ -331,9 +332,12 @@ def subprocess_cmd_and_endpoint(
return (subprocess_cmd, f"localhost:{job_port}")
def prism_arguments(self, job_port) -> typing.List[typing.Any]:
+ # TODO: Remove or expose log_level more nicely.
Review Comment:
TODO before merging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-3078840976
There are tests failing after this PR, could it be related?
https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Xlang_Gcp_Direct.yml?query=event%3Aschedule
https://github.com/apache/beam/actions/workflows/beam_PostCommit_Yaml_Xlang_Direct.yml?query=event%3Aschedule
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205060894
##########
sdks/python/apache_beam/ml/inference/tensorflow_inference_test.py:
##########
@@ -128,7 +128,8 @@ def test_predict_tensor_with_batch_size(self):
model = _create_mult2_model()
model_path = os.path.join(self.tmpdir, f'mult2_{uuid.uuid4()}.keras')
tf.keras.models.save_model(model, model_path)
- with TestPipeline() as pipeline:
+ # FnApiRunner guarantees large batches, which this pipeline assumes
Review Comment:
This could be due to the fact that prism is using accelerated processing time by default. The processing-time timer in BatchElements may not work as expected.
PR #35202 added an experimental flag for using real-time processing time, which may help to resolve this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205071338
##########
sdks/python/apache_beam/runners/direct/direct_runner.py:
##########
@@ -112,9 +118,25 @@ def visit_transform(self, applied_ptransform):
class _PrismRunnerSupportVisitor(PipelineVisitor):
"""Visitor determining if a Pipeline can be run on the PrismRunner."""
- def accept(self, pipeline):
+ def accept(self, pipeline, is_interactive):
+ all_options = options.get_all_options()
self.supported_by_prism_runner = True
- pipeline.visit(self)
+ # TODO(https://github.com/apache/beam/issues/33623): Prism currently
+ # double fires on AfterCount trigger, once appropriately, and once
+ # incorrectly at the end of the window. This if condition could be
+ # more targeted, but for now we'll just ignore all unsafe triggers.
+ if pipeline.allow_unsafe_triggers:
+ self.supported_by_prism_runner = False
+ # TODO(https://github.com/apache/beam/issues/33623): Prism currently
Review Comment:
What else problems do you face here when using prism in interactive mode?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205060894
##########
sdks/python/apache_beam/ml/inference/tensorflow_inference_test.py:
##########
@@ -128,7 +128,8 @@ def test_predict_tensor_with_batch_size(self):
model = _create_mult2_model()
model_path = os.path.join(self.tmpdir, f'mult2_{uuid.uuid4()}.keras')
tf.keras.models.save_model(model, model_path)
- with TestPipeline() as pipeline:
+ # FnApiRunner guarantees large batches, which this pipeline assumes
Review Comment:
This could be due to the fact that prism is using accelerated processing time by default. The processing-time timer in BatchElements may not work as expected.
PR #35202 adds an experimental flag for using real-time processing time, which may help to resolve this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #34612:
URL: https://github.com/apache/beam/pull/34612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #34612:
URL: https://github.com/apache/beam/pull/34612#discussion_r2205100528
##########
sdks/python/apache_beam/transforms/util_test.py:
##########
@@ -254,8 +254,8 @@ def test_constant_batch_no_metrics(self):
self.assertEqual(len(results["distributions"]), 0)
def test_grows_to_max_batch(self):
- # Assumes a single bundle...
- with TestPipeline() as p:
+ # Assumes a single bundle, so we pin to the FnApiRunner
Review Comment:
Do we have a reliable way of getting elements in a single bundle? Maybe only for testing purpose. This seems to be a recurrent issue I met when I am working on some other tests.
Alternatively, we may be able to change the tests so that they won't rely on this assumption, but it is not working all the time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-2992552019
Most remaining flakes should be fixed by https://github.com/apache/beam/pull/35337 with the next release, pausing work until then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable prism by default (where supported) [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34612:
URL: https://github.com/apache/beam/pull/34612#issuecomment-3024754642
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org