You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Miren Esnaola (Jira)" <ji...@apache.org> on 2021/09/07 08:23:00 UTC
[jira] [Updated] (BEAM-12847) Error running Top transform in
Dataflow
[ https://issues.apache.org/jira/browse/BEAM-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Miren Esnaola updated BEAM-12847:
---------------------------------
Description:
When running the following transform in Dataflow (the problem does not happen with the direct running)...
{code:java}
beam.combiners.Top.Of(n=10, key=lambda item: item[1]).without_defaults() {code}
I am getting an error
{code:java}
Error message from worker: generic::unknown: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 284, in _execute response = task() File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 357, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 602, in do_instruction getattr(request, request_type), request.instruction_id) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 639, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 994, in process_bundle element.data) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 222, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 351, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 353, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 921, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 925, in apache_beam.runners.worker.operations.CombineOperation.process File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 835, in extract_only return self.combine_fn.extract_output(accumulator) File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 502, in extract_output heap.sort(reverse=True) File "apache_beam/transforms/cy_combiners.py", line 389, in apache_beam.transforms.cy_combiners.ComparableValue.__lt__ AssertionError passed through: ==> dist_proc/dax/workflow/worker/fnapi_service_impl.cc:644{code}
{{What might be the cause of this issue?}}
was:
When running the following transform in Dataflow (the problem does not happen with the direct running)...
{{}}
{code:java}
beam.combiners.Top.Of(n=10, key=lambda item: item[1]).without_defaults() {code}
{{}}
I am getting an error
{code:java}
Error message from worker: generic::unknown: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 284, in _execute response = task() File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 357, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 602, in do_instruction getattr(request, request_type), request.instruction_id) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 639, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 994, in process_bundle element.data) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 222, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 351, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 353, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 921, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 925, in apache_beam.runners.worker.operations.CombineOperation.process File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 835, in extract_only return self.combine_fn.extract_output(accumulator) File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 502, in extract_output heap.sort(reverse=True) File "apache_beam/transforms/cy_combiners.py", line 389, in apache_beam.transforms.cy_combiners.ComparableValue.__lt__ AssertionError passed through: ==> dist_proc/dax/workflow/worker/fnapi_service_impl.cc:644{code}
{{What might be the cause of this issue?}}
> Error running Top transform in Dataflow
> ---------------------------------------
>
> Key: BEAM-12847
> URL: https://issues.apache.org/jira/browse/BEAM-12847
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Affects Versions: 2.32.0
> Reporter: Miren Esnaola
> Priority: P2
>
> When running the following transform in Dataflow (the problem does not happen with the direct running)...
> {code:java}
> beam.combiners.Top.Of(n=10, key=lambda item: item[1]).without_defaults() {code}
>
> I am getting an error
>
> {code:java}
> Error message from worker: generic::unknown: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 284, in _execute response = task() File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 357, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 602, in do_instruction getattr(request, request_type), request.instruction_id) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 639, in process_bundle bundle_processor.process_bundle(instruction_id)) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 994, in process_bundle element.data) File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 222, in process_encoded self.output(decoded_value) File "apache_beam/runners/worker/operations.py", line 351, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 353, in apache_beam.runners.worker.operations.Operation.output File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive File "apache_beam/runners/worker/operations.py", line 921, in apache_beam.runners.worker.operations.CombineOperation.process File "apache_beam/runners/worker/operations.py", line 925, in apache_beam.runners.worker.operations.CombineOperation.process File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 835, in extract_only return self.combine_fn.extract_output(accumulator) File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 502, in extract_output heap.sort(reverse=True) File "apache_beam/transforms/cy_combiners.py", line 389, in apache_beam.transforms.cy_combiners.ComparableValue.__lt__ AssertionError passed through: ==> dist_proc/dax/workflow/worker/fnapi_service_impl.cc:644{code}
>
> {{What might be the cause of this issue?}}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)