You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/09 13:17:07 UTC
Build failed in Jenkins: beam_PostCommit_Python36 #4337
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4337/display/redirect>
Changes:
------------------------------------------
[...truncated 22.82 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631189784
nanos: 561852216
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631189784
nanos: 580659389
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631189784
nanos: 580864667
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631189784
nanos: 580945730
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631189784
nanos: 581067323
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631189784
nanos: 581611633
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631189784
nanos: 581730604
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.664689779281616 seconds.
INFO:root:Successfully completed job in 5.664689779281616 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:41877
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fba8a739158> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fba8a7391e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fba8a7398c8> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp2_qyq1it/artifacts2uhhq98h' '--job-port' '49227' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36417'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35665'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:49227'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_12017123-5476-434d-912e-0bcc6abea32e.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_12017123-5476-434d-912e-0bcc6abea32e.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_12017123-5476-434d-912e-0bcc6abea32e.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_12017123-5476-434d-912e-0bcc6abea32e.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:30 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0909121630-5dc4d778_a81ae3e8-c5d8-457b-b199-b6a8c0d778aa'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:31 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0909121630-5dc4d778_a81ae3e8-c5d8-457b-b199-b6a8c0d778aa'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:31 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:31 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:32 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:32 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:32 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0909121630-5dc4d778_a81ae3e8-c5d8-457b-b199-b6a8c0d778aa on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40291.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33083.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40047
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0909121630-5dc4d778_a81ae3e8-c5d8-457b-b199-b6a8c0d778aa: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/09 12:16:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0909121630-5dc4d778_a81ae3e8-c5d8-457b-b199-b6a8c0d778aa finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631189795.715179248","description":"Error received from peer ipv4:127.0.0.1:40291","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631189795.715163504","description":"Error received from peer ipv4:127.0.0.1:33083","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631189795.715138095","description":"Error received from peer ipv4:127.0.0.1:40047","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631189795.715138095","description":"Error received from peer ipv4:127.0.0.1:40047","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 16m 33s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2idm3gfgrxu6m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python36 #4355
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4355/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4354
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4354/display/redirect?page=changes>
Changes:
[rohde.samuel] [BEAM-12842] Add timestamp to test work item to deflake
[suztomo] [BEAM-12873] HL7v2IO: to leave schematizedData null, not empty
------------------------------------------
[...truncated 5.61 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-2965-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-2965-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631557223
nanos: 958447694
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631557224
nanos: 91968774
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631557224
nanos: 102657794
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631557224
nanos: 102840900
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631557224
nanos: 102925539
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631557224
nanos: 103017330
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631557224
nanos: 103447914
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631557224
nanos: 103552579
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.702728509902954 seconds.
INFO:root:Successfully completed job in 5.702728509902954 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44551
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6e329c4510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f6e329c4598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6e329c4c80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempoqgusors/artifactsztkar25r' '--job-port' '34559' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42949'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43129'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:34559'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d5d5f93a-7c76-4043-ac2b-1700b1730674.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d5d5f93a-7c76-4043-ac2b-1700b1730674.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d5d5f93a-7c76-4043-ac2b-1700b1730674.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d5d5f93a-7c76-4043-ac2b-1700b1730674.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:29 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0913182029-f8f8bc4f_af7c39dc-1642-420c-a8bf-4c2ba3fdd6a7'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:30 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0913182029-f8f8bc4f_af7c39dc-1642-420c-a8bf-4c2ba3fdd6a7'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:30 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:30 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:31 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0913182029-f8f8bc4f_af7c39dc-1642-420c-a8bf-4c2ba3fdd6a7 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41573.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40199.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40775
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:32 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:32 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0913182029-f8f8bc4f_af7c39dc-1642-420c-a8bf-4c2ba3fdd6a7: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 18:20:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0913182029-f8f8bc4f_af7c39dc-1642-420c-a8bf-4c2ba3fdd6a7 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631557234.388204080","description":"Error received from peer ipv4:127.0.0.1:40775","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631557234.388204080","description":"Error received from peer ipv4:127.0.0.1:40775","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631557234.388204068","description":"Error received from peer ipv4:127.0.0.1:40199","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631557234.388227906","description":"Error received from peer ipv4:127.0.0.1:41573","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 20m 4s
211 actionable tasks: 151 executed, 56 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/dragabz2cmnzw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4353
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4353/display/redirect>
Changes:
------------------------------------------
[...truncated 17.17 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 524463891
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 673557519
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 699134111
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 699365377
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 699473619
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 699574708
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 700874567
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631535686
nanos: 701067924
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
9c43cc2f62b8903a9d21679eb88a59fab0bc35828d4663534d1bb3c824c4c9df
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 23.605668544769287 seconds.
INFO:root:Successfully completed job in 23.605668544769287 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34447
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8db9725510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f8db9725598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8db9725c80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempxt15395v/artifactsl9t5wnk1' '--job-port' '49425' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:49425.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38985'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41405'
WARNING:root:Waiting for grpc channel to be ready at localhost:49425.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:49425'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:49425.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_c658e01b-c79b-4472-8d62-3409de686c10.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_c658e01b-c79b-4472-8d62-3409de686c10.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_c658e01b-c79b-4472-8d62-3409de686c10.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_c658e01b-c79b-4472-8d62-3409de686c10.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:42 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0913122142-a1763f99_f31f7ef0-38ce-4b7b-b794-07815cb5ad0a'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:43 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0913122142-a1763f99_f31f7ef0-38ce-4b7b-b794-07815cb5ad0a'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:45 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:48 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:49 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:49 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0913122142-a1763f99_f31f7ef0-38ce-4b7b-b794-07815cb5ad0a on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33095.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38921.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37771
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0913122142-a1763f99_f31f7ef0-38ce-4b7b-b794-07815cb5ad0a: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.18 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 12:21:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0913122142-a1763f99_f31f7ef0-38ce-4b7b-b794-07815cb5ad0a finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631535714.517789750","description":"Error received from peer ipv4:127.0.0.1:37771","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631535714.517789750","description":"Error received from peer ipv4:127.0.0.1:37771","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631535714.517806492","description":"Error received from peer ipv4:127.0.0.1:33095","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631535714.517803102","description":"Error received from peer ipv4:127.0.0.1:38921","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 21m 26s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/refzgnmi5yt4m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4352
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4352/display/redirect>
Changes:
------------------------------------------
[...truncated 30.64 MB...]
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1700: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1937: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1939: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1969: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1690: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_test.py:1123
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1123: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:181
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:181: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2069
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2069: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2070
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2070: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2083
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2083: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:162
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:162: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:124: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:126
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:126: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:113
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:113: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m== 55 failed, 8 passed, 11 skipped, 180 warnings, 1 error in 1498.91 seconds ===[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 12s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/qmlnil5dgsmsw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4351
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4351/display/redirect>
Changes:
------------------------------------------
[...truncated 30.37 MB...]
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 112599849
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 230427742
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 256520986
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 256731986
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 256824493
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 256927967
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 258408308
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631493265
nanos: 258569002
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
9bbd590c454957e6a9b4a96b3d90cb7fabfc4fa743fe8b15df0aec5b4919f6df
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.850825548171997 seconds.
INFO:root:Successfully completed job in 8.850825548171997 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37127
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fefe5a36510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fefe5a36598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fefe5a36c80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempykos0rjp/artifactsx78tkr88' '--job-port' '33493' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:33493.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:33 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37453'
WARNING:root:Waiting for grpc channel to be ready at localhost:33493.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42233'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33493'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:33493.
WARNING:root:Waiting for grpc channel to be ready at localhost:33493.
WARNING:root:Waiting for grpc channel to be ready at localhost:33493.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_5f4efd34-49a3-4a64-beb0-296a15392cad.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_5f4efd34-49a3-4a64-beb0-296a15392cad.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_5f4efd34-49a3-4a64-beb0-296a15392cad.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_5f4efd34-49a3-4a64-beb0-296a15392cad.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0913003443-d45b6f59_97fdc90d-2542-4dcc-ad27-bdbefd316461'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0913003443-d45b6f59_97fdc90d-2542-4dcc-ad27-bdbefd316461'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:47 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/09/13 00:34:52 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:53 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:54 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0913003443-d45b6f59_97fdc90d-2542-4dcc-ad27-bdbefd316461 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35067.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36335.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38931
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:56 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:56 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0913003443-d45b6f59_97fdc90d-2542-4dcc-ad27-bdbefd316461: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.17 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/13 00:34:57 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0913003443-d45b6f59_97fdc90d-2542-4dcc-ad27-bdbefd316461 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631493298.502994036","description":"Error received from peer ipv4:127.0.0.1:38931","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631493298.502994036","description":"Error received from peer ipv4:127.0.0.1:38931","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631493298.503003957","description":"Error received from peer ipv4:127.0.0.1:35067","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631493298.502999650","description":"Error received from peer ipv4:127.0.0.1:36335","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 34m 27s
211 actionable tasks: 149 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/bx2qx7vcb65mc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4350
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4350/display/redirect>
Changes:
------------------------------------------
[...truncated 13.31 MB...]
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-2965-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 425912618
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 571515083
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 602345466
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 602545499
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 602620601
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 602689504
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 603769540
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631470867
nanos: 603928804
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.769615173339844 seconds.
INFO:root:Successfully completed job in 5.769615173339844 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39093
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0ee3258510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ee3258598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0ee3258c80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp_phbds1c/artifactswfswgsfv' '--job-port' '42615' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34277'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44967'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:42615'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:42615.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9255dda3-f0b8-4e9e-b9b9-e390f69d13f0.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9255dda3-f0b8-4e9e-b9b9-e390f69d13f0.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_9255dda3-f0b8-4e9e-b9b9-e390f69d13f0.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9255dda3-f0b8-4e9e-b9b9-e390f69d13f0.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0912182116-decd893a_fc423b3b-59d7-4ed9-9f4c-69f5f50627a8'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:17 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0912182116-decd893a_fc423b3b-59d7-4ed9-9f4c-69f5f50627a8'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:17 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:18 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:21 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:21 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0912182116-decd893a_fc423b3b-59d7-4ed9-9f4c-69f5f50627a8 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39113.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35397.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35167
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0912182116-decd893a_fc423b3b-59d7-4ed9-9f4c-69f5f50627a8: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.12 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/12 18:21:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0912182116-decd893a_fc423b3b-59d7-4ed9-9f4c-69f5f50627a8 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631470884.918950121","description":"Error received from peer ipv4:127.0.0.1:35167","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631470884.918986103","description":"Error received from peer ipv4:127.0.0.1:39113","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631470884.918950121","description":"Error received from peer ipv4:127.0.0.1:35167","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631470884.918970436","description":"Error received from peer ipv4:127.0.0.1:35397","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 20m 55s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nmx2h6nckckvc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4349
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4349/display/redirect>
Changes:
------------------------------------------
[...truncated 20.60 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631449341
nanos: 913008451
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631449342
nanos: 89098691
}
message: "Renamed 1 shards in 0.18 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631449342
nanos: 125655412
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631449342
nanos: 125847339
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631449342
nanos: 125959873
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631449342
nanos: 126037359
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631449342
nanos: 126461029
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631449342
nanos: 126562118
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 11.766921281814575 seconds.
INFO:root:Successfully completed job in 11.766921281814575 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37927
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f5752c93510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f5752c93598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f5752c93c80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempcn13da49/artifactsyxzs8enh' '--job-port' '59941' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:59941.
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:35 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:44423'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44261'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:59941'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:59941.
WARNING:root:Waiting for grpc channel to be ready at localhost:59941.
WARNING:root:Waiting for grpc channel to be ready at localhost:59941.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1b3b783d-c319-412f-92d9-4b193c20a1c5.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1b3b783d-c319-412f-92d9-4b193c20a1c5.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_1b3b783d-c319-412f-92d9-4b193c20a1c5.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1b3b783d-c319-412f-92d9-4b193c20a1c5.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:41 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0912122241-59344e47_cfa813f8-4e7c-4e2d-9f6a-9117fe66a6f8'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:41 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0912122241-59344e47_cfa813f8-4e7c-4e2d-9f6a-9117fe66a6f8'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:42 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:43 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:47 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:47 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0912122241-59344e47_cfa813f8-4e7c-4e2d-9f6a-9117fe66a6f8 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45303.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44679.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37517
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0912122241-59344e47_cfa813f8-4e7c-4e2d-9f6a-9117fe66a6f8: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.16 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/12 12:22:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0912122241-59344e47_cfa813f8-4e7c-4e2d-9f6a-9117fe66a6f8 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631449370.881980983","description":"Error received from peer ipv4:127.0.0.1:37517","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631449370.881980983","description":"Error received from peer ipv4:127.0.0.1:37517","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631449370.882011658","description":"Error received from peer ipv4:127.0.0.1:44679","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631449370.882023358","description":"Error received from peer ipv4:127.0.0.1:45303","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 22m 25s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/bdio52z7desp4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4348
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4348/display/redirect>
Changes:
------------------------------------------
[...truncated 30.37 MB...]
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1700: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1937: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1939: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1969: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1690: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_test.py:1123
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1123: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2069
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2069: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2070
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2070: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2083
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2083: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:181
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:181: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:124: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:113
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:113: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:162
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:162: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:126
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:126: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m== 46 failed, 17 passed, 11 skipped, 182 warnings, 1 error in 1656.81 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 47s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/zjgxz62gyuf3c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4347
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4347/display/redirect>
Changes:
------------------------------------------
[...truncated 36.95 MB...]
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1700: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1937: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1939: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1969: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_test.py:1123
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1123: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1690: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:124: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:181
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:181: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:162
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:162: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:113
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:113: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:126
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:126: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2069
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2069: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2070
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2070: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2083
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2083: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m== 44 failed, 19 passed, 11 skipped, 188 warnings, 1 error in 2608.60 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 48m 1s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/cvml54gfwnxpy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4346
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4346/display/redirect>
Changes:
------------------------------------------
[...truncated 24.57 MB...]
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 504434585
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 621515035
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 649346590
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 649525403
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 649597644
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 649667978
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 650695323
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631384283
nanos: 650876522
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.275166273117065 seconds.
INFO:root:Successfully completed job in 6.275166273117065 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44305
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb261525510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb261525598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb261525c80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temph3in1wir/artifactsx391y4kb' '--job-port' '33085' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33081'
WARNING:root:Waiting for grpc channel to be ready at localhost:33085.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34179'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33085'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:33085.
WARNING:root:Waiting for grpc channel to be ready at localhost:33085.
WARNING:root:Waiting for grpc channel to be ready at localhost:33085.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_5e47411f-16f1-4308-a20c-69d1236109f0.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_5e47411f-16f1-4308-a20c-69d1236109f0.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_5e47411f-16f1-4308-a20c-69d1236109f0.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_5e47411f-16f1-4308-a20c-69d1236109f0.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0911181818-a3b0b475_3486e70a-7297-4b2d-a84b-dc3603f18cf7'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0911181818-a3b0b475_3486e70a-7297-4b2d-a84b-dc3603f18cf7'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:19 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:20 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:23 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:23 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0911181818-a3b0b475_3486e70a-7297-4b2d-a84b-dc3603f18cf7 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37047.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34109.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36187
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0911181818-a3b0b475_3486e70a-7297-4b2d-a84b-dc3603f18cf7: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.15 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 18:18:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0911181818-a3b0b475_3486e70a-7297-4b2d-a84b-dc3603f18cf7 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631384307.747936081","description":"Error received from peer ipv4:127.0.0.1:36187","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631384307.747936081","description":"Error received from peer ipv4:127.0.0.1:36187","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631384307.747957029","description":"Error received from peer ipv4:127.0.0.1:34109","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631384307.747976456","description":"Error received from peer ipv4:127.0.0.1:37047","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 46m 31s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/raefq6jor43h2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4345
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4345/display/redirect>
Changes:
------------------------------------------
[...truncated 13.14 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 308855056
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 475045680
}
message: "Renamed 1 shards in 0.17 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 484735727
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 484947443
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 485024690
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 485096693
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 486038208
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631362732
nanos: 486186027
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.008282423019409 seconds.
INFO:root:Successfully completed job in 6.008282423019409 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38993
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3fa5418510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3fa5418598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3fa5418c80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempsmkaz1ra/artifacts6vivla2b' '--job-port' '44683' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:18:59 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42517'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:00 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41755'
WARNING:root:Waiting for grpc channel to be ready at localhost:44683.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:00 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:44683'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:00 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_105f8423-7065-4b4e-8859-50f779ec1b6d.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_105f8423-7065-4b4e-8859-50f779ec1b6d.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_105f8423-7065-4b4e-8859-50f779ec1b6d.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_105f8423-7065-4b4e-8859-50f779ec1b6d.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:02 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0911121902-5c906863_09ca494d-0f42-4863-abc2-5b52e9095a61'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:02 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0911121902-5c906863_09ca494d-0f42-4863-abc2-5b52e9095a61'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:03 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:05 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:08 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:08 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0911121902-5c906863_09ca494d-0f42-4863-abc2-5b52e9095a61 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42427.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45203.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40263
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0911121902-5c906863_09ca494d-0f42-4863-abc2-5b52e9095a61: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 12:19:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0911121902-5c906863_09ca494d-0f42-4863-abc2-5b52e9095a61 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631362752.348323568","description":"Error received from peer ipv4:127.0.0.1:40263","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631362752.348323568","description":"Error received from peer ipv4:127.0.0.1:40263","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631362752.348324147","description":"Error received from peer ipv4:127.0.0.1:45203","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631362752.348345686","description":"Error received from peer ipv4:127.0.0.1:42427","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 45m 45s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mniktsutbur5a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4344
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4344/display/redirect>
Changes:
------------------------------------------
[...truncated 17.78 MB...]
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631341572
nanos: 878617286
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631341573
nanos: 83438634
}
message: "Renamed 1 shards in 0.20 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631341573
nanos: 91865301
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631341573
nanos: 92116594
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631341573
nanos: 92263460
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631341573
nanos: 92400789
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631341573
nanos: 93861818
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631341573
nanos: 94040155
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 10.47146463394165 seconds.
INFO:root:Successfully completed job in 10.47146463394165 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38465
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fdb064fc510> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fdb064fc598> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fdb064fcc80> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temptnoweev5/artifactsxrftf0dj' '--job-port' '40973' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38199'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42511'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:40973'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1c887949-d4a0-4605-ad25-771f7e70677e.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1c887949-d4a0-4605-ad25-771f7e70677e.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_1c887949-d4a0-4605-ad25-771f7e70677e.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1c887949-d4a0-4605-ad25-771f7e70677e.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:23 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0911062623-940e6d81_66b911a4-bc6f-4ca2-be4f-53ae780ee567'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:23 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0911062623-940e6d81_66b911a4-bc6f-4ca2-be4f-53ae780ee567'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:24 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:24 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:25 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:25 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:25 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0911062623-940e6d81_66b911a4-bc6f-4ca2-be4f-53ae780ee567 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42007.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36981.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45889
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0911062623-940e6d81_66b911a4-bc6f-4ca2-be4f-53ae780ee567: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/11 06:26:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0911062623-940e6d81_66b911a4-bc6f-4ca2-be4f-53ae780ee567 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631341588.146844872","description":"Error received from peer ipv4:127.0.0.1:36981","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631341588.146824222","description":"Error received from peer ipv4:127.0.0.1:45889","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631341588.146824222","description":"Error received from peer ipv4:127.0.0.1:45889","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631341588.146846336","description":"Error received from peer ipv4:127.0.0.1:42007","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 6m 49s
211 actionable tasks: 160 executed, 47 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vjqre54yjxtha
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4343
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4343/display/redirect?page=changes>
Changes:
[noreply] Port changes from Pub/Sub Lite to beam (#15418)
[heejong] [BEAM-12805] Fix XLang CombinePerKey test by explicitly assigning the
[BenWhitehead] [BEAM-8376] Google Cloud Firestore Connector - Add handling for
[noreply] Decreasing peak memory usage for beam.TupleCombineFn (#15494)
[noreply] [BEAM-12802] Add support for prefetch through data layers down through
[noreply] [BEAM-11097] Add implementation of side input cache (#15483)
------------------------------------------
[...truncated 20.60 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-MapTuple-collect_values-_19"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:start_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Group-RestoreTags_20"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:element_count:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AQ==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_13"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Match_22"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Group-RestoreTags_20"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:element_count:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AQ==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_15"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Unkey_21"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:start_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Match_22"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:finish_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Match_22"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:finish_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "fn/read/ref_PCollection_PCollection_11:0"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:finish_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Group-RestoreTags_20"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:start_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Unkey_21"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:element_count:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AQ==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_14"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "fn/read/ref_PCollection_PCollection_11:0"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:ptransform_execution_time:total_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Match_22"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AA==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PTRANSFORM": "ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-MapTuple-collect_values-_19"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AY4CjgKOAg==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_11"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AYoCigKKAg==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_12"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AQ4ODg==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_15"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AYQChAKEAg==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_13"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "payload": "AfwB/AH8AQ==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' "PCOLLECTION": "ref_PCollection_PCollection_14"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'}'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: FileChannelManager removed spill file directory /tmp/flink-io-dc490a6c-271b-456e-9564-bc1b316a299c'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.io.network.NettyShuffleEnvironment close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Shutting down the network environment and its components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: FileChannelManager removed spill file directory /tmp/flink-netty-shuffle-c3f0ac26-279e-4440-aa47-cd64948dc5b8'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.taskexecutor.KvStateService shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Shutting down the kvState service and its components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Stop job leader service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.filecache.FileCache shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: removed file cache directory /tmp/flink-dist-cache-70e6d61c-c8dd-4756-ba97-a676f3471713'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.taskexecutor.TaskExecutor handleOnStopException'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Stopped TaskExecutor akka://flink/user/rpc/taskmanager_0.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Stopping Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Stopping Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$7'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Stopped Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Shutting down BLOB cache'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Shutting down BLOB cache'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.blob.BlobServer close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Stopped BLOB server at 0.0.0.0:33197'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Sep 11, 2021 12:27:26 AM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$7'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Stopped Akka RPC service.'
PASSED [100%]
=============================== warnings summary ===============================
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_iobase_source
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_iobase_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1937: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_iobase_source
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_iobase_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1939: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_iobase_source
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_iobase_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1969: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-flink-py36.xml> -
============== 14 passed, 4 skipped, 6 warnings in 668.09 seconds ==============
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1631319717.315118193","description":"Error received from peer ipv4:127.0.0.1:32827","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT
[gw2] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_iobase_source
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 32m 51s
211 actionable tasks: 181 executed, 26 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iu5mcdn3gj7ys
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4342
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4342/display/redirect?page=changes>
Changes:
[noreply] Added type annotations to some combiners missing it. (#15414)
[noreply] [BEAM-12634] JmsIO auto scaling feature (#15464)
[noreply] [BEAM-12662] Get Flink version from cluster. (#15223)
------------------------------------------
[...truncated 14.26 MB...]
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 458020210
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 584181785
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 603533506
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 603733539
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 603809595
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 603890895
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 604402780
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631297894
nanos: 604545831
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 23.29563069343567 seconds.
INFO:root:Successfully completed job in 23.29563069343567 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38637
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fdd9b678400> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fdd9b678488> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fdd9b678b70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp3p_ie6mb/artifacts7dixi_7v' '--job-port' '55003' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:55003.
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36179'
WARNING:root:Waiting for grpc channel to be ready at localhost:55003.
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:32943'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:55003'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:55003.
WARNING:root:Waiting for grpc channel to be ready at localhost:55003.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:28 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_cddae504-3b98-4180-b634-695e0e73a175.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:28 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_cddae504-3b98-4180-b634-695e0e73a175.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:28 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_cddae504-3b98-4180-b634-695e0e73a175.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:28 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_cddae504-3b98-4180-b634-695e0e73a175.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:28 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0910181828-38860a17_8d65d6da-f8ae-40f1-80ac-ea9276336e18'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:29 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0910181828-38860a17_8d65d6da-f8ae-40f1-80ac-ea9276336e18'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:29 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:32 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:36 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:36 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0910181828-38860a17_8d65d6da-f8ae-40f1-80ac-ea9276336e18 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44667.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33253.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41761
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0910181828-38860a17_8d65d6da-f8ae-40f1-80ac-ea9276336e18: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.11 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/10 18:18:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0910181828-38860a17_8d65d6da-f8ae-40f1-80ac-ea9276336e18 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631297920.950339899","description":"Error received from peer ipv4:127.0.0.1:44667","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631297920.950338729","description":"Error received from peer ipv4:127.0.0.1:33253","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631297920.950340893","description":"Error received from peer ipv4:127.0.0.1:41761","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1631297920.950340893","description":"Error received from peer ipv4:127.0.0.1:41761","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 36m 34s
211 actionable tasks: 156 executed, 51 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3fyglhssipwb4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4341
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4341/display/redirect?page=changes>
Changes:
[noreply] Register MapCoder, some comments/cleanup. (#15471)
[noreply] [BEAM-12588] Multimap user state proto changes (#15473)
------------------------------------------
[...truncated 29.58 MB...]
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1969: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1690: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
apache_beam/io/gcp/bigquery.py:1694
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1694: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1700: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:63
apache_beam/io/gcp/tests/utils.py:63
apache_beam/io/gcp/tests/utils.py:63
apache_beam/io/gcp/tests/utils.py:63
apache_beam/io/gcp/tests/utils.py:63
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/io/gcp/bigquery_test.py:1123
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1123: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:181
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:181: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2069
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2069: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2070
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2070: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2083
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2083: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:162
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:162: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:126
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:126: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:124: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:113
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:113: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m====== 11 failed, 52 passed, 11 skipped, 176 warnings in 5573.52 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 48m 39s
211 actionable tasks: 172 executed, 35 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5npf57p56tdlu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4340
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4340/display/redirect?page=changes>
Changes:
[ruwan.lambrichts] Clarify additional_bq_parameters argument
[esert] Bump a throttling counter on BigQueryRead retries due to
[noreply] [BEAM-5097] Increment counter for "small words" in go SDK example
------------------------------------------
[...truncated 29.31 MB...]
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
apache_beam/io/gcp/bigquery.py:1700
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1700: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
apache_beam/io/gcp/bigquery.py:1937
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1937: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
apache_beam/io/gcp/bigquery.py:1939
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1939: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
apache_beam/io/gcp/bigquery.py:1969
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1969: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
apache_beam/io/gcp/bigquery.py:1690
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1690: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
apache_beam/dataframe/io.py:569
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1123
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1123: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:181
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:181: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2069
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2069: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2070
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2070: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2083
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2083: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:162
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:162: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:126
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:126: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:124: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:113
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:113: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 4 failed, 59 passed, 11 skipped, 178 warnings in 6853.34 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 51s
211 actionable tasks: 150 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/bk25km3p2yqqq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4339
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4339/display/redirect?page=changes>
Changes:
[noreply] Fix broken 'differences from pandas' link
[noreply] Added GroupBy row in Aggregation table.
[Luke Cwik] [BEAM-12769] Fix typo in test class name, CLass -> Class
[Etienne Chauchot] [BEAM-5172] Temporary ignore testSplit and testSizes tests waiting for a
[samuelw] [BEAM-12740] Remove matching to filter files when renaming gcs files in
[noreply] [BEAM-3304] Helper functions for triggers (#15430)
------------------------------------------
[...truncated 145.50 KB...]
WARNING:root:Waiting for grpc channel to be ready at localhost:41891.
WARNING:root:Waiting for grpc channel to be ready at localhost:41891.
WARNING:root:Waiting for grpc channel to be ready at localhost:41891.
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
WARNING:root:Waiting for grpc channel to be ready at localhost:41891.
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:44 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36867'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:44 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42001'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:45 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:41891'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:45 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:41891.
WARNING:root:Waiting for grpc channel to be ready at localhost:41891.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_72fbdcf3-acad-4a9d-a0ed-f3964e4ca600.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_72fbdcf3-acad-4a9d-a0ed-f3964e4ca600.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_72fbdcf3-acad-4a9d-a0ed-f3964e4ca600.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_72fbdcf3-acad-4a9d-a0ed-f3964e4ca600.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:52 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0910002452-48bb9587_f4ed5ee0-6c99-4bd3-a5ea-24bb13f38abf'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:53 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0910002452-48bb9587_f4ed5ee0-6c99-4bd3-a5ea-24bb13f38abf'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:55 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:24:57 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:04 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:04 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0910002452-48bb9587_f4ed5ee0-6c99-4bd3-a5ea-24bb13f38abf on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41405.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40437.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:32969
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:10 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0910002452-48bb9587_f4ed5ee0-6c99-4bd3-a5ea-24bb13f38abf: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.16 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/10 00:25:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0910002452-48bb9587_f4ed5ee0-6c99-4bd3-a5ea-24bb13f38abf finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631233514.874956286","description":"Error received from peer ipv4:127.0.0.1:41405","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631233514.874083542","description":"Error received from peer ipv4:127.0.0.1:40437","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631233514.870335484","description":"Error received from peer ipv4:127.0.0.1:32969","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631233514.870335484","description":"Error received from peer ipv4:127.0.0.1:32969","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36IT
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md
warning: check: missing required meta-data: url
warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
> Task :sdks:python:test-suites:portable:py36:postCommitPy36IT
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Runner closed connection"
debug_error_string = "{"created":"@1631233571.692265792","description":"Error received from peer ipv4:127.0.0.1:43377","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Runner closed connection","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1631233731.356847366","description":"Error received from peer ipv4:127.0.0.1:37159","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1631234183.595635583","description":"Error received from peer ipv4:127.0.0.1:36605","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36IT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 41m 14s
211 actionable tasks: 182 executed, 25 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ndbuh5tug5btc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4338
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4338/display/redirect?page=changes>
Changes:
[kawaigin] [BEAM-10708] Support streaming cache in beam_sql magic
------------------------------------------
[...truncated 12.90 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1631214430
nanos: 674924612
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631214431
nanos: 169544935
}
message: "Renamed 1 shards in 0.49 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1631214431
nanos: 218022823
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631214431
nanos: 218220710
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631214431
nanos: 218298673
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631214431
nanos: 218389511
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631214431
nanos: 218862056
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1631214431
nanos: 218974351
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
f5eabcac298936114f40ede25ad01469c0abf6cf082ebc62de87ee527a53e54a
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 63.960208892822266 seconds.
INFO:root:Successfully completed job in 63.960208892822266 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35245
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.34.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd0989ea158> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd0989ea1e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd0989ea8c8> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.34.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempph5u0qn4/artifactsw4_xopsd' '--job-port' '44855' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:44855.
WARNING:root:Waiting for grpc channel to be ready at localhost:44855.
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46423'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44291'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:44855'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:44855.
WARNING:root:Waiting for grpc channel to be ready at localhost:44855.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d660ee3d-6583-4286-9642-00724b484f0e.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d660ee3d-6583-4286-9642-00724b484f0e.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d660ee3d-6583-4286-9642-00724b484f0e.null.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d660ee3d-6583-4286-9642-00724b484f0e.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:39 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0909190739-dd2c5f17_0bfde430-5e86-4ae6-aebd-e4b1a2e68026'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:39 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0909190739-dd2c5f17_0bfde430-5e86-4ae6-aebd-e4b1a2e68026'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:40 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:07:53 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:03 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:03 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0909190739-dd2c5f17_0bfde430-5e86-4ae6-aebd-e4b1a2e68026 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41647.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34675.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:42521
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:08 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:08 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0909190739-dd2c5f17_0bfde430-5e86-4ae6-aebd-e4b1a2e68026: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.23 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/09/09 19:08:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0909190739-dd2c5f17_0bfde430-5e86-4ae6-aebd-e4b1a2e68026 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631214492.817309577","description":"Error received from peer ipv4:127.0.0.1:34675","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631214492.817275679","description":"Error received from peer ipv4:127.0.0.1:42521","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1631214492.817309493","description":"Error received from peer ipv4:127.0.0.1:41647","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 7m 45s
211 actionable tasks: 160 executed, 47 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/q2qrlvjlewsgo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org