You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/10/27 02:23:22 UTC
Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #8557
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8557/display/redirect?page=changes>
Changes:
[baeminbo] [BEAM-6721] Set numShards dynamically for TextIO.write()
[melissapa] [BEAM-11758] Update basics page: Splittable DoFn
[noreply] Change sql.Options to an interface under sqlx. (#15790)
[melissapa] Address review feedback
[noreply] [BEAM-4149] Ensure that we always provide and require the worker id.
[noreply] [BEAM-13098] Fix translation of repeated TableRow fields (#15779)
------------------------------------------
[...truncated 20.05 MB...]
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction
return getattr(self, request_type)(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15864']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T02:10:34.350Z: JOB_MESSAGE_ERROR: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -15908: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction
return getattr(self, request_type)(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15898']
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:60)
org.apache.beam.runners.dataflow.worker.fn.control.RegisterAndProcessBundleOperation.finish(RegisterAndProcessBundleOperation.java:338)
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:103)
org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1437)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$1100(StreamingDataflowWorker.java:165)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$7.run(StreamingDataflowWorker.java:1113)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -15908: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction
return getattr(self, request_type)(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15898']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-10-26_17_55_52-17811697331907041987 is in state JOB_STATE_CANCELLING
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py38-xdist.xml> -
[31m[1m======== 1 failed, 29 passed, 3 skipped, 8 warnings in 5269.17 seconds =========[0m
> Task :sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests FAILED
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 22m 48s
92 actionable tasks: 74 executed, 18 from cache
Publishing build scan...
https://gradle.com/s/f7co423q7vtbe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_VR_Dataflow
#8561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8561/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #8560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8560/display/redirect?page=changes>
Changes:
[relax] support json
[Andrew Pilloud] [BEAM-13118] Don't force push to master
[noreply] Merge pull request #15731: [BEAM-13067] Mark GroupIntoBatches output as
------------------------------------------
[...truncated 20.05 MB...]
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15783']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:06.544Z: JOB_MESSAGE_ERROR: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -15827: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction
return getattr(self, request_type)(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15823']
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:60)
org.apache.beam.runners.dataflow.worker.fn.control.RegisterAndProcessBundleOperation.finish(RegisterAndProcessBundleOperation.java:338)
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:103)
org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1437)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$1100(StreamingDataflowWorker.java:165)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$7.run(StreamingDataflowWorker.java:1113)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -15827: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction
return getattr(self, request_type)(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15823']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.436Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-10-27_11_47_30-6315732531014595250.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.459Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-min-globally/Create/Impulse+multiple-combines/assert-min-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-min-globally/Create/Map(decode)+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.459Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/Values+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.460Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/CombinePerKey/GroupByKey/ReadStream+multiple-combines/min-globally/CombinePerKey/GroupByKey/MergeBuckets+multiple-combines/min-globally/CombinePerKey/Combine+multiple-combines/min-globally/UnKey+multiple-combines/min-globally/InjectDefault/MapToVoidKey0+multiple-combines/min-globally/InjectDefault/MapToVoidKey0+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.460Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-count-globally/Group/RestoreTags+multiple-combines/assert-count-globally/Unkey+multiple-combines/assert-count-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.460Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-largest-globally/Group/RestoreTags+multiple-combines/assert-largest-globally/Unkey+multiple-combines/assert-largest-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.460Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/largest-globally/Top(2)/Create/Impulse+multiple-combines/largest-globally/Top(2)/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/largest-globally/Top(2)/Create/Map(decode)+multiple-combines/largest-globally/Top(2)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.460Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/ReadStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/MergeBuckets+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/UnKey+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/MapToVoidKey0+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/MapToVoidKey0+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.461Z: JOB_MESSAGE_BASIC: Finished operation Create/Impulse+Create/FlatMap(<lambda at core.py:3222>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.461Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/Impulse+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/FlatMap(<lambda at core.py:3222>)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/Map(decode)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/InjectDefault+multiple-combines/assert-count-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-count-globally/ToVoidKey+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.461Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-largest-globally/Create/Impulse+multiple-combines/assert-largest-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-largest-globally/Create/Map(decode)+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.461Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/ReadStream+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/MergeBuckets+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Create/Map(decode)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/KeyWithVoid+multiple-combines/largest-globally/Top(2)/ParDo(_TopPerBundle)+multiple-combines/largest-globally/Top(2)/GroupByKey/WriteStream+multiple-combines/min-globally/KeyWithVoid+multiple-combines/min-globally/CombinePerKey/GroupByKey/WriteStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.461Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/Values+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.462Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-min-globally/Group/RestoreTags+multiple-combines/assert-min-globally/Unkey+multiple-combines/assert-min-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.462Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/DoOnce/Impulse+multiple-combines/min-globally/DoOnce/FlatMap(<lambda at core.py:3222>)+multiple-combines/min-globally/DoOnce/Map(decode)+multiple-combines/min-globally/InjectDefault/InjectDefault+multiple-combines/assert-min-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-min-globally/ToVoidKey+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.462Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/largest-globally/Top(2)/GroupByKey/ReadStream+multiple-combines/largest-globally/Top(2)/GroupByKey/MergeBuckets+multiple-combines/largest-globally/Top(2)/ParDo(_MergeTopPerBundle)+multiple-combines/assert-largest-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-largest-globally/ToVoidKey+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.462Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-count-globally/Create/Impulse+multiple-combines/assert-count-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-count-globally/Create/Map(decode)+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.578Z: JOB_MESSAGE_DETAILED: Cleaning up.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.656Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T20:02:10.680Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-10-27_11_47_30-6315732531014595250 is in state JOB_STATE_CANCELLING
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py38-xdist.xml> -
[31m[1m======== 1 failed, 29 passed, 3 skipped, 8 warnings in 5307.07 seconds =========[0m
> Task :sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests FAILED
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 14m 40s
92 actionable tasks: 66 executed, 26 from cache
Publishing build scan...
https://gradle.com/s/x6fjil7r5tnje
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #8559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8559/display/redirect?page=changes>
Changes:
[mmack] [BEAM-8958] Use AWS credentials provider with BasicKinesisProvider (AWS
------------------------------------------
[...truncated 20.14 MB...]
element.data)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15896']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.681Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-10-27_05_41_28-17211155000630242574.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.709Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-min-globally/Create/Impulse+multiple-combines/assert-min-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-min-globally/Create/Map(decode)+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.709Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/Impulse+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/FlatMap(<lambda at core.py:3222>)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/Map(decode)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/InjectDefault+multiple-combines/assert-count-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-count-globally/ToVoidKey+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.710Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-largest-globally/Create/Impulse+multiple-combines/assert-largest-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-largest-globally/Create/Map(decode)+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.710Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/CombinePerKey/GroupByKey/ReadStream+multiple-combines/min-globally/CombinePerKey/GroupByKey/MergeBuckets+multiple-combines/min-globally/CombinePerKey/Combine+multiple-combines/min-globally/UnKey+multiple-combines/min-globally/InjectDefault/MapToVoidKey0+multiple-combines/min-globally/InjectDefault/MapToVoidKey0+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.710Z: JOB_MESSAGE_BASIC: Finished operation Create/Impulse+Create/FlatMap(<lambda at core.py:3222>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.710Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/ReadStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/MergeBuckets+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/UnKey+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/MapToVoidKey0+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/MapToVoidKey0+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/largest-globally/Top(2)/Create/Impulse+multiple-combines/largest-globally/Top(2)/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/largest-globally/Top(2)/Create/Map(decode)+multiple-combines/largest-globally/Top(2)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-count-globally/Create/Impulse+multiple-combines/assert-count-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-count-globally/Create/Map(decode)+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-largest-globally/Group/RestoreTags+multiple-combines/assert-largest-globally/Unkey+multiple-combines/assert-largest-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/largest-globally/Top(2)/GroupByKey/ReadStream+multiple-combines/largest-globally/Top(2)/GroupByKey/MergeBuckets+multiple-combines/largest-globally/Top(2)/ParDo(_MergeTopPerBundle)+multiple-combines/assert-largest-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-largest-globally/ToVoidKey+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/Values+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/Values+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-count-globally/Group/RestoreTags+multiple-combines/assert-count-globally/Unkey+multiple-combines/assert-count-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.729Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/ReadStream+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/MergeBuckets+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Create/Map(decode)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/KeyWithVoid+multiple-combines/largest-globally/Top(2)/ParDo(_TopPerBundle)+multiple-combines/largest-globally/Top(2)/GroupByKey/WriteStream+multiple-combines/min-globally/KeyWithVoid+multiple-combines/min-globally/CombinePerKey/GroupByKey/WriteStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.730Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/DoOnce/Impulse+multiple-combines/min-globally/DoOnce/FlatMap(<lambda at core.py:3222>)+multiple-combines/min-globally/DoOnce/Map(decode)+multiple-combines/min-globally/InjectDefault/InjectDefault+multiple-combines/assert-min-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-min-globally/ToVoidKey+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.730Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-min-globally/Group/RestoreTags+multiple-combines/assert-min-globally/Unkey+multiple-combines/assert-min-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.855Z: JOB_MESSAGE_DETAILED: Cleaning up.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.907Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:15.944Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T13:56:16.949Z: JOB_MESSAGE_ERROR: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -15942: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 581, in do_instruction
getattr(request, request_type), request.instruction_id)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 996, in process_bundle
element.data)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15936']
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:60)
org.apache.beam.runners.dataflow.worker.fn.control.RegisterAndProcessBundleOperation.finish(RegisterAndProcessBundleOperation.java:338)
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:103)
org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1437)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$1100(StreamingDataflowWorker.java:165)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$7.run(StreamingDataflowWorker.java:1113)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -15942: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 581, in do_instruction
getattr(request, request_type), request.instruction_id)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 996, in process_bundle
element.data)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.7/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15936']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-10-27_05_41_28-17211155000630242574 is in state JOB_STATE_CANCELLING
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py37-xdist.xml> -
[31m[1m============== 1 failed, 29 passed, 3 skipped in 5363.32 seconds ===============[0m
> Task :sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests FAILED
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 9m 58s
92 actionable tasks: 62 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/tunhpujh2gkzs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #8558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8558/display/redirect?page=changes>
Changes:
[heejong] [BEAM-12978] Customizable dependency for Java external transform
[heejong] add ConfigT param to getDependencies method
------------------------------------------
[...truncated 20.16 MB...]
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-15988']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.287Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-10-27_00_00_42-6433819882592616925.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/Values+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-largest-globally/Group/RestoreTags+multiple-combines/assert-largest-globally/Unkey+multiple-combines/assert-largest-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-count-globally/Create/Impulse+multiple-combines/assert-count-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-count-globally/Create/Map(decode)+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-min-globally/Create/Impulse+multiple-combines/assert-min-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-min-globally/Create/Map(decode)+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-min-globally/Group/RestoreTags+multiple-combines/assert-min-globally/Unkey+multiple-combines/assert-min-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/largest-globally/Top(2)/GroupByKey/ReadStream+multiple-combines/largest-globally/Top(2)/GroupByKey/MergeBuckets+multiple-combines/largest-globally/Top(2)/ParDo(_MergeTopPerBundle)+multiple-combines/assert-largest-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-largest-globally/ToVoidKey+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/ReadStream+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/MergeBuckets+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/Values+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/StreamingPCollectionViewWriter
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/ReadStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/MergeBuckets+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/Combine+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/UnKey+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/MapToVoidKey0+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/MapToVoidKey0+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.307Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/Impulse+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/FlatMap(<lambda at core.py:3222>)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/DoOnce/Map(decode)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/InjectDefault/InjectDefault+multiple-combines/assert-count-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-count-globally/ToVoidKey+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.308Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-largest-globally/Create/Impulse+multiple-combines/assert-largest-globally/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/assert-largest-globally/Create/Map(decode)+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/Tag[0]+multiple-combines/assert-largest-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.308Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/CombinePerKey/GroupByKey/ReadStream+multiple-combines/min-globally/CombinePerKey/GroupByKey/MergeBuckets+multiple-combines/min-globally/CombinePerKey/Combine+multiple-combines/min-globally/UnKey+multiple-combines/min-globally/InjectDefault/MapToVoidKey0+multiple-combines/min-globally/InjectDefault/MapToVoidKey0+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/PairWithVoidKey+multiple-combines/min-globally/InjectDefault/_DataflowIterableAsMultimapSideInput(MapToVoidKey0.out.0)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.308Z: JOB_MESSAGE_BASIC: Finished operation Create/Impulse+Create/FlatMap(<lambda at core.py:3222>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.308Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/ReadStream+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/GroupByKey/MergeBuckets+multiple-combines/assert-count-globally/Group/CoGroupByKeyImpl/MapTuple(collect_values)+multiple-combines/assert-count-globally/Group/RestoreTags+multiple-combines/assert-count-globally/Unkey+multiple-combines/assert-count-globally/Match
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.308Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/ReadStream+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/MergeBuckets+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Create/Map(decode)+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/KeyWithVoid+multiple-combines/largest-globally/Top(2)/ParDo(_TopPerBundle)+multiple-combines/largest-globally/Top(2)/GroupByKey/WriteStream+multiple-combines/min-globally/KeyWithVoid+multiple-combines/min-globally/CombinePerKey/GroupByKey/WriteStream+multiple-combines/count-globally/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.308Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/min-globally/DoOnce/Impulse+multiple-combines/min-globally/DoOnce/FlatMap(<lambda at core.py:3222>)+multiple-combines/min-globally/DoOnce/Map(decode)+multiple-combines/min-globally/InjectDefault/InjectDefault+multiple-combines/assert-min-globally/WindowInto(WindowIntoFn)+multiple-combines/assert-min-globally/ToVoidKey+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/Tag[1]+multiple-combines/assert-min-globally/Group/CoGroupByKeyImpl/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.308Z: JOB_MESSAGE_BASIC: Finished operation multiple-combines/largest-globally/Top(2)/Create/Impulse+multiple-combines/largest-globally/Top(2)/Create/FlatMap(<lambda at core.py:3222>)+multiple-combines/largest-globally/Top(2)/Create/Map(decode)+multiple-combines/largest-globally/Top(2)/GroupByKey/WriteStream
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.458Z: JOB_MESSAGE_DETAILED: Cleaning up.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.504Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:33.535Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T08:15:34.558Z: JOB_MESSAGE_ERROR: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -16032: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction
return getattr(self, request_type)(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-16028']
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:60)
org.apache.beam.runners.dataflow.worker.fn.control.RegisterAndProcessBundleOperation.finish(RegisterAndProcessBundleOperation.java:338)
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:103)
org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1437)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$1100(StreamingDataflowWorker.java:165)
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$7.run(StreamingDataflowWorker.java:1113)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -16032: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
response = task()
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 580, in do_instruction
return getattr(self, request_type)(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 995, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 347, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 349, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 216, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 536, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1334, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/usr/local/lib/python3.8/site-packages/apache_beam/transforms/combiners.py", line 383, in process
for element in reversed(bundle):
TypeError: 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible [while running 'generatedPtransform-16028']
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:180)
org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:765)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-10-27_00_00_42-6433819882592616925 is in state JOB_STATE_CANCELLING
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py38-xdist.xml> -
[31m[1m======== 1 failed, 29 passed, 3 skipped, 8 warnings in 5250.76 seconds =========[0m
> Task :sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests FAILED
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 149
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerBatchTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 27m 15s
92 actionable tasks: 76 executed, 16 from cache
Publishing build scan...
https://gradle.com/s/wcmwmanxrpzlc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org