You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Valentyn Tymofieiev (JIRA)" <ji...@apache.org> on 2019/06/12 23:17:00 UTC
[jira] [Comment Edited] (BEAM-7548)
test_approximate_unique_global_by_error is flaky
[ https://issues.apache.org/jira/browse/BEAM-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862540#comment-16862540 ]
Valentyn Tymofieiev edited comment on BEAM-7548 at 6/12/19 11:16 PM:
---------------------------------------------------------------------
I think the rootcause is non-determinism of hash() in Python. We may want to consider this test as inherently flaky and a retry decorator akin to https://github.com/apache/beam/blob/2f9d7112bc03b94c419eb48805972f7792d57b7d/sdks/python/apache_beam/runners/worker/statesampler_test.py#L41. Alternatively, tests should supply a deterministic hash function to ApproximateUniqueTest.
was (Author: tvalentyn):
I think the rootcause is non-determinism of hash() in Python. We may want to consider this test as inherently flaky and a retry decorator akin to https://github.com/apache/beam/blob/2f9d7112bc03b94c419eb48805972f7792d57b7d/sdks/python/apache_beam/runners/worker/statesampler_test.py#L41. Alternatively, tests should supply a deterministic hash function.
> test_approximate_unique_global_by_error is flaky
> ------------------------------------------------
>
> Key: BEAM-7548
> URL: https://issues.apache.org/jira/browse/BEAM-7548
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
> Reporter: Valentyn Tymofieiev
> Assignee: Hannah Jiang
> Priority: Major
>
> The error happened on Jenkins in Python 3.5 suite, which currently uses Python 3.5.2 interpreter:
> {noformat}
> 11:57:47 ======================================================================
> 11:57:47 ERROR: test_approximate_unique_global_by_error (apache_beam.transforms.stats_test.ApproximateUniqueTest)
> 11:57:47 ----------------------------------------------------------------------
> 11:57:47 Traceback (most recent call last):
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/transforms/stats_test.py", line 236, in test_approximate_unique_global_by_error
> 11:57:47 pipeline.run()
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/testing/test_pipeline.py", line 107, in run
> 11:57:47 else test_runner_api))
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/pipeline.py", line 406, in run
> 11:57:47 self._options).run(False)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/pipeline.py", line 419, in run
> 11:57:47 return self.runner.run_pipeline(self, self._options)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/direct/direct_runner.py", line 128, in run_pipeline
> 11:57:47 return runner.run_pipeline(pipeline, options)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 289, in run_pipeline
> 11:57:47 default_environment=self._default_environment))
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 293, in run_via_runner_api
> 11:57:47 return self.run_stages(*self.create_stages(pipeline_proto))
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 369, in run_stages
> 11:57:47 stage_context.safe_coders)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 531, in run_stage
> 11:57:47 data_input, data_output)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 1235, in process_bundle
> 11:57:47 result_future = self._controller.control_handler.push(process_bundle)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 851, in push
> 11:57:47 response = self.worker.do_instruction(request)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 342, in do_instruction
> 11:57:47 request.instruction_id)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 368, in process_bundle
> 11:57:47 bundle_processor.process_bundle(instruction_id))
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 595, in process_bundle
> 11:57:47 data.ptransform_id].process_encoded(data.data)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 143, in process_encoded
> 11:57:47 self.output(decoded_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 256, in output
> 11:57:47 cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 143, in receive
> 11:57:47 self.consumer.process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 594, in process
> 11:57:47 delayed_application = self.dofn_receiver.receive(o)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 778, in receive
> 11:57:47 self.process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 784, in process
> 11:57:47 self._reraise_augmented(exn)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 782, in process
> 11:57:47 return self.do_fn_invoker.invoke_process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 595, in invoke_process
> 11:57:47 windowed_value, additional_args, additional_kwargs, output_processor)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 666, in _invoke_process_per_window
> 11:57:47 windowed_value, self.process_method(*args_for_process))
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 921, in process_outputs
> 11:57:47 self.main_receivers.receive(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 143, in receive
> 11:57:47 self.consumer.process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 594, in process
> 11:57:47 delayed_application = self.dofn_receiver.receive(o)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 778, in receive
> 11:57:47 self.process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 784, in process
> 11:57:47 self._reraise_augmented(exn)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 782, in process
> 11:57:47 return self.do_fn_invoker.invoke_process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 454, in invoke_process
> 11:57:47 windowed_value, self.process_method(windowed_value.value))
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 921, in process_outputs
> 11:57:47 self.main_receivers.receive(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 143, in receive
> 11:57:47 self.consumer.process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 594, in process
> 11:57:47 delayed_application = self.dofn_receiver.receive(o)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 778, in receive
> 11:57:47 self.process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 784, in process
> 11:57:47 self._reraise_augmented(exn)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 851, in _reraise_augmented
> 11:57:47 raise_with_traceback(new_exn)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/target/.tox-py35/py35/lib/python3.5/site-packages/future/utils/__init__.py", line 419, in raise_with_traceback
> 11:57:47 raise exc.with_traceback(traceback)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 782, in process
> 11:57:47 return self.do_fn_invoker.invoke_process(windowed_value)
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 454, in invoke_process
> 11:57:47 windowed_value, self.process_method(windowed_value.value))
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/transforms/core.py", line 1293, in <lambda>
> 11:57:47 wrapper = lambda x: [fn(x)]
> 11:57:47 File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/testing/util.py", line 130, in _equal
> 11:57:47 'Failed assert: %r == %r' % (sorted_expected, sorted_actual))
> 11:57:47 apache_beam.testing.util.BeamAssertException: Failed assert: [True] == [False] [while running 'assert:global_by_error/Match']
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)