You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/08/11 07:55:46 UTC
Build failed in Jenkins: beam_PostCommit_Python38 #1542
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1542/display/redirect>
Changes:
------------------------------------------
[...truncated 195.42 KB...]
INFO:root:severity: INFO
timestamp {
seconds: 1628663101
nanos: 367291927
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628663101
nanos: 367478132
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628663101
nanos: 367558002
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628663101
nanos: 367624759
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628663101
nanos: 368099689
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628663101
nanos: 368214607
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 38.87765693664551 seconds.
INFO:root:Successfully completed job in 38.87765693664551 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42349
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4d1f628550> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f4d1f6285e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4d1f628d30> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp3fqq66hd/artifactsa2qlnyzb' '--job-port' '52511' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:52511.
WARNING:root:Waiting for grpc channel to be ready at localhost:52511.
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37553'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34759'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:52511'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:52511.
WARNING:root:Waiting for grpc channel to be ready at localhost:52511.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_5c5dcc4f-402d-43e3-b0ec-146925518971.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_5c5dcc4f-402d-43e3-b0ec-146925518971.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_5c5dcc4f-402d-43e3-b0ec-146925518971.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_5c5dcc4f-402d-43e3-b0ec-146925518971.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:30 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0811062530-66382c3_c1ac97d7-1111-4956-9b0b-66a87b678b06'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:30 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0811062530-66382c3_c1ac97d7-1111-4956-9b0b-66a87b678b06'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:31 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:33 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/08/11 06:25:37 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:37 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:38 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0811062530-66382c3_c1ac97d7-1111-4956-9b0b-66a87b678b06 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40213.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41333.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:32921
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:40 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:40 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0811062530-66382c3_c1ac97d7-1111-4956-9b0b-66a87b678b06: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.05 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/11 06:25:41 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0811062530-66382c3_c1ac97d7-1111-4956-9b0b-66a87b678b06 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628663142.336906744","description":"Error received from peer ipv4:127.0.0.1:32921","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628663142.336926036","description":"Error received from peer ipv4:127.0.0.1:40213","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628663142.336926210","description":"Error received from peer ipv4:127.0.0.1:41333","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628663142.336906744","description":"Error received from peer ipv4:127.0.0.1:32921","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 146
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 16s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/32dzckonmrqrc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python38 #1600
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1600/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1599
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1599/display/redirect>
Changes:
------------------------------------------
[...truncated 1.18 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 191 warnings in 6598.25 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 27s
208 actionable tasks: 144 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/kv2whytbqe3ji
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1598
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1598/display/redirect?page=changes>
Changes:
[Robert Burke] [BEAM-9615] Report Row as Known URN
[noreply] Merge pull request #15357 from [BEAM-12781] Add memoization of
------------------------------------------
[...truncated 37.46 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 195 warnings in 5960.58 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 44m 17s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/m6au3v3ope6mm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1597
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1597/display/redirect?page=changes>
Changes:
[ihor.indyk] Decreasing peak memory usage for
[ihor.indyk] Changing the default `merge_accumulators_batch_size`
[noreply] Clarify PCollection immutability. (#15227)
[noreply] [BEAM-11218] ptest allows to obtain a pipeline result (#15364)
[Kyle Weaver] Remove trailing whitespace
------------------------------------------
[...truncated 31.22 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629850853
nanos: 965024232
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629850854
nanos: 97727060
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629850854
nanos: 109120845
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629850854
nanos: 109327316
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629850854
nanos: 109489202
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629850854
nanos: 109603643
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629850854
nanos: 111308336
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629850854
nanos: 111439943
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 11.385528802871704 seconds.
INFO:root:Successfully completed job in 11.385528802871704 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34051
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7efed7608ca0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7efed7608d30> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7efed76074c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp3ax8u7mm/artifactsiiq9oajz' '--job-port' '46709' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:46709.
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46849'
WARNING:root:Waiting for grpc channel to be ready at localhost:46709.
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43303'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:46709'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:46709.
WARNING:root:Waiting for grpc channel to be ready at localhost:46709.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_48baab4e-9364-47cd-ad3a-d602341048d7.'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_48baab4e-9364-47cd-ad3a-d602341048d7.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_48baab4e-9364-47cd-ad3a-d602341048d7.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_48baab4e-9364-47cd-ad3a-d602341048d7.'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0825002108-ec719050_3be0ca7f-ed99-40c6-a973-25195529cbca'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0825002108-ec719050_3be0ca7f-ed99-40c6-a973-25195529cbca'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:09 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:11 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:15 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:15 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0825002108-ec719050_3be0ca7f-ed99-40c6-a973-25195529cbca on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35399.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40279.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36355
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0825002108-ec719050_3be0ca7f-ed99-40c6-a973-25195529cbca: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/25 00:21:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0825002108-ec719050_3be0ca7f-ed99-40c6-a973-25195529cbca finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629850879.955998238","description":"Error received from peer ipv4:127.0.0.1:40279","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629850879.955973048","description":"Error received from peer ipv4:127.0.0.1:36355","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629850879.956035404","description":"Error received from peer ipv4:127.0.0.1:35399","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629850879.955973048","description":"Error received from peer ipv4:127.0.0.1:36355","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 21s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/kai4gzemg72v6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1596
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1596/display/redirect?page=changes>
Changes:
[noreply] [Go SDK] Go SDK Exits Experimental (#15374)
[noreply] [BEAM-12724][BEAM-12349] Support user timers in Samza portable runner
------------------------------------------
[...truncated 25.03 MB...]
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 604262113
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 646427631
}
message: "Renamed 1 shards in 0.04 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 665897846
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 666158437
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 666309833
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 666431188
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 668030500
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629829193
nanos: 668239593
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 12.614797830581665 seconds.
INFO:root:Successfully completed job in 12.614797830581665 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35849
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4f714ac940> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f4f714ac9d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4f71431160> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp4bv5r5_t/artifactspttyz19g' '--job-port' '51781' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:51781.
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35881'
WARNING:root:Waiting for grpc channel to be ready at localhost:51781.
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:45751'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:51781'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:51781.
WARNING:root:Waiting for grpc channel to be ready at localhost:51781.
WARNING:root:Waiting for grpc channel to be ready at localhost:51781.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_977de983-4848-4f33-acd7-8548f25a6a8a.'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_977de983-4848-4f33-acd7-8548f25a6a8a.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_977de983-4848-4f33-acd7-8548f25a6a8a.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_977de983-4848-4f33-acd7-8548f25a6a8a.'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:09 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0824182009-8f4f626d_00876469-8faa-4645-8f4e-c2154921c41f'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:09 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0824182009-8f4f626d_00876469-8faa-4645-8f4e-c2154921c41f'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:15 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:19 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:20 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0824182009-8f4f626d_00876469-8faa-4645-8f4e-c2154921c41f on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39553.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36049.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37101
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0824182009-8f4f626d_00876469-8faa-4645-8f4e-c2154921c41f: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/24 18:20:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0824182009-8f4f626d_00876469-8faa-4645-8f4e-c2154921c41f finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629829223.952039619","description":"Error received from peer ipv4:127.0.0.1:37101","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629829223.952083620","description":"Error received from peer ipv4:127.0.0.1:39553","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629829223.952041856","description":"Error received from peer ipv4:127.0.0.1:36049","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629829223.952039619","description":"Error received from peer ipv4:127.0.0.1:37101","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 42s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/rcdnhgpvrqeyy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1595
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1595/display/redirect>
Changes:
------------------------------------------
[...truncated 898.28 KB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 187 warnings in 6618.04 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:container:java8:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 11s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/zzlvgxoqhe5mw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1594
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1594/display/redirect>
Changes:
------------------------------------------
[...truncated 19.14 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 193 warnings in 6645.61 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 40s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/edfqh5ujexgh2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1593
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1593/display/redirect?page=changes>
Changes:
[Andrew Pilloud] [BEAM-9379] Remove Java11 specific SQL timeout
[noreply] [BEAM-8571] Release Go SDK containers, and use them. (#15365)
[noreply] [BEAM-6374] Elide collecting unnecessary pcollection metrics (#15358)
------------------------------------------
[...truncated 37.48 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 191 warnings in 6573.95 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 48s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xllcx6srtpua4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1592
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1592/display/redirect>
Changes:
------------------------------------------
[...truncated 37.49 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 195 warnings in 6777.70 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 22s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4fe2tz46bk4di
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1591
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1591/display/redirect>
Changes:
------------------------------------------
[...truncated 37.52 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 193 warnings in 5706.91 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 44s
212 actionable tasks: 150 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/gyhosb6otrxuo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1590
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1590/display/redirect>
Changes:
------------------------------------------
[...truncated 32.61 MB...]
thread: "Thread-14"
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 491522312
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 548707008
}
message: "Renamed 1 shards in 0.06 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 555046319
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 555249929
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 555328845
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 555448293
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 556071281
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629699843
nanos: 556224107
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.019087076187134 seconds.
INFO:root:Successfully completed job in 6.019087076187134 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39465
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa2e1e74940> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa2e1e749d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa2e1df9160> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temptfy4xlr7/artifactsm6bpd78e' '--job-port' '33555' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:08 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45279'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:08 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:36019'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:08 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33555'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:08 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_37e17f4f-d1e8-4340-a095-9235dcd6536c.'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_37e17f4f-d1e8-4340-a095-9235dcd6536c.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_37e17f4f-d1e8-4340-a095-9235dcd6536c.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_37e17f4f-d1e8-4340-a095-9235dcd6536c.'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:09 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0823062409-70c5f94_d086beae-d0a2-47fc-ad11-f6b401fff0de'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:09 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0823062409-70c5f94_d086beae-d0a2-47fc-ad11-f6b401fff0de'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:10 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:11 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:11 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0823062409-70c5f94_d086beae-d0a2-47fc-ad11-f6b401fff0de on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38985.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40387.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:42521
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:12 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0823062409-70c5f94_d086beae-d0a2-47fc-ad11-f6b401fff0de: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/23 06:24:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0823062409-70c5f94_d086beae-d0a2-47fc-ad11-f6b401fff0de finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629699854.278891013","description":"Error received from peer ipv4:127.0.0.1:40387","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629699854.278905919","description":"Error received from peer ipv4:127.0.0.1:38985","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629699854.278863204","description":"Error received from peer ipv4:127.0.0.1:42521","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629699854.278863204","description":"Error received from peer ipv4:127.0.0.1:42521","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 30s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/yzsudf3ji64ge
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1589
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1589/display/redirect>
Changes:
------------------------------------------
[...truncated 19.98 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 195 warnings in 5507.58 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 34m 2s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/j2fnhe34o7ms2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1588
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1588/display/redirect>
Changes:
------------------------------------------
[...truncated 24.60 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 839475631
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 926898717
}
message: "Renamed 1 shards in 0.09 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 936486005
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 936646223
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 936716794
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 936781167
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 938018083
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629670869
nanos: 938142776
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.006663084030151 seconds.
INFO:root:Successfully completed job in 7.006663084030151 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39583
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f08d0db0940> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f08d0db09d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f08d0d33160> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp3gfn84d1/artifactspaf1m7b4' '--job-port' '59449' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:40775'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43493'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:17 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:59449'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:17 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:59449.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_ed1083fd-f128-48d5-b257-c85374d5c42f.'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_ed1083fd-f128-48d5-b257-c85374d5c42f.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_ed1083fd-f128-48d5-b257-c85374d5c42f.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_ed1083fd-f128-48d5-b257-c85374d5c42f.'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:19 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0822222119-f1a40cf3_a1cc0482-16c1-4524-a8b9-00a7307068f2'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:19 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0822222119-f1a40cf3_a1cc0482-16c1-4524-a8b9-00a7307068f2'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:20 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:21 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:24 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:24 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0822222119-f1a40cf3_a1cc0482-16c1-4524-a8b9-00a7307068f2 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45501.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35751.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41355
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0822222119-f1a40cf3_a1cc0482-16c1-4524-a8b9-00a7307068f2: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.05 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/22 22:21:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0822222119-f1a40cf3_a1cc0482-16c1-4524-a8b9-00a7307068f2 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629670888.389107669","description":"Error received from peer ipv4:127.0.0.1:45501","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629670888.389083011","description":"Error received from peer ipv4:127.0.0.1:41355","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629670888.389097040","description":"Error received from peer ipv4:127.0.0.1:35751","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629670888.389083011","description":"Error received from peer ipv4:127.0.0.1:41355","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 35m 31s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/uccrb7nziyxyg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1587
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1587/display/redirect>
Changes:
------------------------------------------
[...truncated 19.83 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 187 warnings in 6608.04 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 7s
212 actionable tasks: 161 executed, 47 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/44rwgbs4kxf5e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1586
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1586/display/redirect>
Changes:
------------------------------------------
[...truncated 37.10 MB...]
apache_beam/io/gcp/bigquery.py:1974
apache_beam/io/gcp/bigquery.py:1974
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1974: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 197 warnings in 6651.99 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py38:docker'.
> Process 'command 'docker'' finished with non-zero exit value 2
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 59s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7djhpgvabbkgi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1585
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1585/display/redirect>
Changes:
------------------------------------------
[...truncated 37.45 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 3 failed, 63 passed, 8 skipped, 187 warnings in 5577.51 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 37m 13s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/qouparn7dmjuo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1584
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1584/display/redirect>
Changes:
------------------------------------------
[...truncated 37.47 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
apache_beam/io/gcp/bigquery.py:2242
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2242: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
apache_beam/io/gcp/bigquery.py:2244
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2244: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
apache_beam/io/gcp/bigquery.py:2268
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2268: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
apache_beam/io/gcp/bigquery.py:1964
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1964: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:440
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:440: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:554
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:554: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2383
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2383: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2384
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2384: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2397
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2397: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 195 warnings in 6756.51 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 43s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/gzfl4fgaejcys
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1583
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1583/display/redirect>
Changes:
------------------------------------------
[...truncated 24.64 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-2965-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 366235733
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 408862113
}
message: "Renamed 1 shards in 0.04 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 427784204
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 428061246
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 428216457
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 428345918
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 430611848
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629548669
nanos: 430845260
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 22.698546648025513 seconds.
INFO:root:Successfully completed job in 22.698546648025513 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35811
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe7bd95b940> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe7bd95b9d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe7bd8e0160> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp6si5_5y7/artifactsz_goybmb' '--job-port' '56831' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36801'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42669'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56831'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:56831.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_c203b5be-200b-494e-a17a-cd99d6bf90a0.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_c203b5be-200b-494e-a17a-cd99d6bf90a0.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_c203b5be-200b-494e-a17a-cd99d6bf90a0.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_c203b5be-200b-494e-a17a-cd99d6bf90a0.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:38 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0821122438-51303690_3c12f7f9-9a32-45fa-97e4-8aaebbc10e64'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:39 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0821122438-51303690_3c12f7f9-9a32-45fa-97e4-8aaebbc10e64'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:39 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:39 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:40 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:40 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0821122438-51303690_3c12f7f9-9a32-45fa-97e4-8aaebbc10e64 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35233.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38785.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44249
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0821122438-51303690_3c12f7f9-9a32-45fa-97e4-8aaebbc10e64: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 12:24:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0821122438-51303690_3c12f7f9-9a32-45fa-97e4-8aaebbc10e64 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629548683.998396238","description":"Error received from peer ipv4:127.0.0.1:35233","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629548683.998332911","description":"Error received from peer ipv4:127.0.0.1:44249","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629548683.998351511","description":"Error received from peer ipv4:127.0.0.1:38785","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629548683.998332911","description":"Error received from peer ipv4:127.0.0.1:44249","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 19s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/hu2npdmedkn6g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1582
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1582/display/redirect?page=changes>
Changes:
[noreply] Avoid spamming lull logging (#15366)
------------------------------------------
[...truncated 24.65 MB...]
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 519073963
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 612438678
}
message: "Renamed 1 shards in 0.09 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 642151832
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 642513513
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 642642259
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 642771959
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:907"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 643998861
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629527284
nanos: 644293546
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 37.054731130599976 seconds.
INFO:root:Successfully completed job in 37.054731130599976 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35885
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f74baf03940> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f74baf039d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f74bae88160> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temphjmm5uv5/artifacts0aigelo7' '--job-port' '53413' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:53413.
WARNING:root:Waiting for grpc channel to be ready at localhost:53413.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38147'
WARNING:root:Waiting for grpc channel to be ready at localhost:53413.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43837'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:53413'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:53413.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_bba00152-6155-483e-98d0-42c67cc1db68.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_bba00152-6155-483e-98d0-42c67cc1db68.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_bba00152-6155-483e-98d0-42c67cc1db68.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_bba00152-6155-483e-98d0-42c67cc1db68.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0821062827-c92c00ce_5fac29a9-e7dd-432e-acdc-bca416fabbb3'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:27 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0821062827-c92c00ce_5fac29a9-e7dd-432e-acdc-bca416fabbb3'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:28 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:30 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/08/21 06:28:34 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:35 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:35 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0821062827-c92c00ce_5fac29a9-e7dd-432e-acdc-bca416fabbb3 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35339.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44033.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45707
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:38 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:38 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0821062827-c92c00ce_5fac29a9-e7dd-432e-acdc-bca416fabbb3: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.03 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 06:28:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0821062827-c92c00ce_5fac29a9-e7dd-432e-acdc-bca416fabbb3 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1038, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629527320.601507109","description":"Error received from peer ipv4:127.0.0.1:44033","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629527320.601549140","description":"Error received from peer ipv4:127.0.0.1:35339","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629527320.601486176","description":"Error received from peer ipv4:127.0.0.1:45707","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629527320.601486176","description":"Error received from peer ipv4:127.0.0.1:45707","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 43m 47s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wn5zgaxmj2r2i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1581
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1581/display/redirect?page=changes>
Changes:
[zhoufek] [BEAM-9487] Disable allowing unsafe triggers by default
[noreply] [BEAM-10917] Add support for BigQuery Read API in Python BEAM (#15185)
[ryanthompson591] Change filter to also retry on 408 errors.
------------------------------------------
[...truncated 24.64 MB...]
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 207625389
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 243357658
}
message: "Renamed 1 shards in 0.04 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 258336305
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 258563280
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 258658647
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 258733272
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 259450674
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629505619
nanos: 259587764
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 17.20519781112671 seconds.
INFO:root:Successfully completed job in 17.20519781112671 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42005
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0dd394a940> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0dd394a9d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0dd38cf160> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempna3yv4k3/artifactszo11p226' '--job-port' '52199' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:52199.
WARNING:root:Waiting for grpc channel to be ready at localhost:52199.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:08 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:40033'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:38725'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:52199'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:52199.
WARNING:root:Waiting for grpc channel to be ready at localhost:52199.
WARNING:root:Waiting for grpc channel to be ready at localhost:52199.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_fcdcbefe-daee-4c91-b02e-33acca412831.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_fcdcbefe-daee-4c91-b02e-33acca412831.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_fcdcbefe-daee-4c91-b02e-33acca412831.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_fcdcbefe-daee-4c91-b02e-33acca412831.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:15 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0821002715-c07d9444_9ddbf0d3-0716-4e03-ab6c-b541097d7c07'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:16 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0821002715-c07d9444_9ddbf0d3-0716-4e03-ab6c-b541097d7c07'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:17 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:19 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:23 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:23 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0821002715-c07d9444_9ddbf0d3-0716-4e03-ab6c-b541097d7c07 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39043.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35977.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34211
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:26 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0821002715-c07d9444_9ddbf0d3-0716-4e03-ab6c-b541097d7c07: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/21 00:27:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0821002715-c07d9444_9ddbf0d3-0716-4e03-ab6c-b541097d7c07 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629505648.560304511","description":"Error received from peer ipv4:127.0.0.1:39043","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629505648.560208063","description":"Error received from peer ipv4:127.0.0.1:34211","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629505648.560246299","description":"Error received from peer ipv4:127.0.0.1:35977","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629505648.560208063","description":"Error received from peer ipv4:127.0.0.1:34211","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 37m 13s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/g7bn6w335enmi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1580
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1580/display/redirect?page=changes>
Changes:
[samuelw] [BEAM-12780] StreamingDataflowWorker should only retry exceptions
------------------------------------------
[...truncated 24.55 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 240588188
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 299059391
}
message: "Renamed 1 shards in 0.06 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 304800271
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 304980754
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 305063009
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 305135965
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 306124687
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629483493
nanos: 306264877
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.859560251235962 seconds.
INFO:root:Successfully completed job in 7.859560251235962 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:41797
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f401e5ebc10> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f401e5ebca0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f401e5f0430> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-templcd0upnu/artifactsjiz4hn5w' '--job-port' '35249' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:35249.
WARNING:root:Waiting for grpc channel to be ready at localhost:35249.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43729'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39229'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:35249'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:35249.
WARNING:root:Waiting for grpc channel to be ready at localhost:35249.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_39f0d551-bd28-4026-b686-d8562ee5f54a.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_39f0d551-bd28-4026-b686-d8562ee5f54a.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_39f0d551-bd28-4026-b686-d8562ee5f54a.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_39f0d551-bd28-4026-b686-d8562ee5f54a.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0820181827-21c99b41_256011c6-df17-439b-8a65-e82ae21ce9f3'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:27 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0820181827-21c99b41_256011c6-df17-439b-8a65-e82ae21ce9f3'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:28 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:30 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:35 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:35 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0820181827-21c99b41_256011c6-df17-439b-8a65-e82ae21ce9f3 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33631.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45633.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38845
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:37 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:37 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0820181827-21c99b41_256011c6-df17-439b-8a65-e82ae21ce9f3: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 18:18:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0820181827-21c99b41_256011c6-df17-439b-8a65-e82ae21ce9f3 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629483519.270682587","description":"Error received from peer ipv4:127.0.0.1:33631","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629483519.270615979","description":"Error received from peer ipv4:127.0.0.1:38845","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629483519.270666388","description":"Error received from peer ipv4:127.0.0.1:45633","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629483519.270615979","description":"Error received from peer ipv4:127.0.0.1:38845","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 42s
212 actionable tasks: 159 executed, 49 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/za3xb7h54mc3q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python38 - Build # 1579 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python38 - Build # 1579 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python38/1579/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python38 #1578
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1578/display/redirect>
Changes:
------------------------------------------
[...truncated 22.43 MB...]
thread: "Thread-14"
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-2968-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-2968-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 418576240
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 457090854
}
message: "Renamed 1 shards in 0.04 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 470958232
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 471207380
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 471305370
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 471391677
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 472059965
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629440612
nanos: 472193956
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.72730016708374 seconds.
INFO:root:Successfully completed job in 7.72730016708374 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:46487
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd1f44efc10> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd1f44efca0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd1f44f4430> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempa_lgecvs/artifacts2ii7cdnw' '--job-port' '53089' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38495'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33179'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:53089'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1290d802-6bcd-4707-a3d4-8e0c27e94ddb.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1290d802-6bcd-4707-a3d4-8e0c27e94ddb.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_1290d802-6bcd-4707-a3d4-8e0c27e94ddb.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1290d802-6bcd-4707-a3d4-8e0c27e94ddb.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:38 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0820062338-511b8f15_78f6d682-7a36-43ff-929b-290d8011ea06'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:38 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0820062338-511b8f15_78f6d682-7a36-43ff-929b-290d8011ea06'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:39 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:39 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:40 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:40 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0820062338-511b8f15_78f6d682-7a36-43ff-929b-290d8011ea06 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44239.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33187.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41801
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0820062338-511b8f15_78f6d682-7a36-43ff-929b-290d8011ea06: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 06:23:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0820062338-511b8f15_78f6d682-7a36-43ff-929b-290d8011ea06 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629440623.797010674","description":"Error received from peer ipv4:127.0.0.1:41801","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629440623.797044511","description":"Error received from peer ipv4:127.0.0.1:33187","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629440623.797059547","description":"Error received from peer ipv4:127.0.0.1:44239","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629440623.797010674","description":"Error received from peer ipv4:127.0.0.1:41801","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 38m 32s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iki3wfolku6zw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1577
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1577/display/redirect?page=changes>
Changes:
[noreply] Change conflicting StateReader name from side to reader (#15348)
[noreply] [BEAM-3304] Go triggering support (#15239)
------------------------------------------
[...truncated 26.94 MB...]
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629419334
nanos: 913166046
}
message: "Renamed 1 shards in 0.02 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629419334
nanos: 921253204
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629419334
nanos: 921412467
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629419334
nanos: 921487569
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629419334
nanos: 921581745
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629419334
nanos: 922366857
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629419334
nanos: 922492980
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 16.55047917366028 seconds.
INFO:root:Successfully completed job in 16.55047917366028 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42329
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f932cdebc10> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f932cdebca0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f932cdf0430> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp_uvc92dj/artifacts0jwnfckq' '--job-port' '60735' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:60735.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46121'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44241'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:60735'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:60735.
WARNING:root:Waiting for grpc channel to be ready at localhost:60735.
WARNING:root:Waiting for grpc channel to be ready at localhost:60735.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d17125e3-365b-42b0-a736-77e8b6d425c4.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d17125e3-365b-42b0-a736-77e8b6d425c4.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d17125e3-365b-42b0-a736-77e8b6d425c4.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d17125e3-365b-42b0-a736-77e8b6d425c4.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:15 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0820002915-9c6d7da1_17221f9d-829b-4eb8-a3f5-75af564e39c6'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:15 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0820002915-9c6d7da1_17221f9d-829b-4eb8-a3f5-75af564e39c6'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:16 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:18 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:21 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:21 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0820002915-9c6d7da1_17221f9d-829b-4eb8-a3f5-75af564e39c6 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41041.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36399.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34977
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:24 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0820002915-9c6d7da1_17221f9d-829b-4eb8-a3f5-75af564e39c6: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.13 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/20 00:29:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0820002915-9c6d7da1_17221f9d-829b-4eb8-a3f5-75af564e39c6 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629419366.964445546","description":"Error received from peer ipv4:127.0.0.1:34977","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629419366.964469467","description":"Error received from peer ipv4:127.0.0.1:36399","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629419366.964482350","description":"Error received from peer ipv4:127.0.0.1:41041","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629419366.964445546","description":"Error received from peer ipv4:127.0.0.1:34977","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 29s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/pp6q7iimufawa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python38 - Build # 1576 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python38 - Build # 1576 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python38/1576/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python38 #1575
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1575/display/redirect>
Changes:
------------------------------------------
[...truncated 24.75 MB...]
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629376425
nanos: 528694629
}
message: "Renamed 1 shards in 0.09 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629376425
nanos: 536314725
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629376425
nanos: 536469936
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629376425
nanos: 536542415
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629376425
nanos: 536605834
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629376425
nanos: 537042617
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629376425
nanos: 537149190
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 14.81845211982727 seconds.
INFO:root:Successfully completed job in 14.81845211982727 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38681
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f10e5393e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f10e5393ee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f10e5399670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp1sdg54pg/artifacts1xyx_dz3' '--job-port' '56603' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:56603.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:33:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45179'
WARNING:root:Waiting for grpc channel to be ready at localhost:56603.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:33:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44003'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:33:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56603'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:33:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:56603.
WARNING:root:Waiting for grpc channel to be ready at localhost:56603.
WARNING:root:Waiting for grpc channel to be ready at localhost:56603.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a7e07cdc-18f5-4a09-87cf-5e5a053b7a05.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a7e07cdc-18f5-4a09-87cf-5e5a053b7a05.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a7e07cdc-18f5-4a09-87cf-5e5a053b7a05.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a7e07cdc-18f5-4a09-87cf-5e5a053b7a05.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:01 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0819123400-9b15e75f_c5130012-c00a-4aef-8b35-4a259636f165'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:01 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0819123400-9b15e75f_c5130012-c00a-4aef-8b35-4a259636f165'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:02 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:03 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:07 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:07 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0819123400-9b15e75f_c5130012-c00a-4aef-8b35-4a259636f165 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43323.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38341.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37757
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0819123400-9b15e75f_c5130012-c00a-4aef-8b35-4a259636f165: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.03 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 12:34:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0819123400-9b15e75f_c5130012-c00a-4aef-8b35-4a259636f165 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629376451.994019191","description":"Error received from peer ipv4:127.0.0.1:38341","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629376451.994010729","description":"Error received from peer ipv4:127.0.0.1:43323","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629376451.993954692","description":"Error received from peer ipv4:127.0.0.1:37757","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629376451.993954692","description":"Error received from peer ipv4:127.0.0.1:37757","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 4s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/n5upni4xiznk6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1574
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1574/display/redirect>
Changes:
------------------------------------------
[...truncated 22.87 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629354460
nanos: 855260133
}
message: "Renamed 1 shards in 0.07 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629354460
nanos: 863465309
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629354460
nanos: 863688230
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629354460
nanos: 863779544
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629354460
nanos: 863880395
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629354460
nanos: 864608287
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629354460
nanos: 864762783
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 35.5272319316864 seconds.
INFO:root:Successfully completed job in 35.5272319316864 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42321
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7cfc289e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7cfc289ee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7cfc28f670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempww7dgwnm/artifacts5ysmxekl' '--job-port' '42475' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:42475.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:27:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41041'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:27:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46381'
WARNING:root:Waiting for grpc channel to be ready at localhost:42475.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:27:59 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:42475'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:27:59 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:42475.
WARNING:root:Waiting for grpc channel to be ready at localhost:42475.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_80777c51-26a3-4ffd-9e84-0dc63e711417.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_80777c51-26a3-4ffd-9e84-0dc63e711417.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_80777c51-26a3-4ffd-9e84-0dc63e711417.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_80777c51-26a3-4ffd-9e84-0dc63e711417.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:03 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0819062803-c9b57086_a6ccff61-c607-4197-b94f-6127e7007ec5'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:04 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0819062803-c9b57086_a6ccff61-c607-4197-b94f-6127e7007ec5'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:04 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:07 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:10 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:10 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0819062803-c9b57086_a6ccff61-c607-4197-b94f-6127e7007ec5 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:46213.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36201.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40161
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0819062803-c9b57086_a6ccff61-c607-4197-b94f-6127e7007ec5: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.02 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/19 06:28:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0819062803-c9b57086_a6ccff61-c607-4197-b94f-6127e7007ec5 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629354495.103440334","description":"Error received from peer ipv4:127.0.0.1:40161","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629354495.103802565","description":"Error received from peer ipv4:127.0.0.1:36201","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629354495.103862139","description":"Error received from peer ipv4:127.0.0.1:46213","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629354495.103440334","description":"Error received from peer ipv4:127.0.0.1:40161","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 40s
212 actionable tasks: 150 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ucpw7uow5fylc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python38 - Build # 1573 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python38 - Build # 1573 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python38/1573/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python38 #1572
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1572/display/redirect?page=changes>
Changes:
[samuelw] [BEAM-12740] [BEAM-8268] Improve error handling and retries for GCS
[aromanenko.dev] [BEAM-12270] Add Parquet source support for TPD-DS benchmark
[noreply] [BEAM-12742] Samza Runner does not properly delete modified timer
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-12 (beam) in workspace <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git --version # 'git version 2.7.4'
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a0fbe00ef12b72ec89672ab32ccc6d6331ca5edd (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a0fbe00ef12b72ec89672ab32ccc6d6331ca5edd # timeout=10
Commit message: "Merge pull request #15301: [BEAM-12740] [BEAM-8268] Improve error handling and retries for GCS rename used by FileBasedSink"
> git rev-list --no-walk 410ad7699621e28433d81809f6b9c42fe7bd6a60 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib
[EnvInject] - Variables injected successfully.
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/gradlew> --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :python38PostCommit
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=f70ac48f-808a-4631-b8f1-5812b9d97449, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 26320
log file: /home/jenkins/.gradle/daemon/6.8.3/daemon-26320.out.log
----- Last 20 lines from daemon log file - daemon-26320.out.log -----
2021-08-18T18:00:30.505+0000 [INFO] [org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler] Received command: Build{id=f70ac48f-808a-4631-b8f1-5812b9d97449, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src}.>
2021-08-18T18:00:30.505+0000 [DEBUG] [org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler] Starting executing command: Build{id=f70ac48f-808a-4631-b8f1-5812b9d97449, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src}> with connection: socket connection from /127.0.0.1:33467 to /127.0.0.1:59724.
2021-08-18T18:00:30.506+0000 [ERROR] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] Command execution: started DaemonCommandExecution[command = Build{id=f70ac48f-808a-4631-b8f1-5812b9d97449, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src},> connection = DefaultDaemonConnection: socket connection from /127.0.0.1:33467 to /127.0.0.1:59724] after 0.0 minutes of idle
2021-08-18T18:00:30.507+0000 [INFO] [org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Marking the daemon as busy, address: [3813aa70-f51b-483f-a21e-42deeb54fdea port:33467, addresses:[localhost/127.0.0.1]]
2021-08-18T18:00:30.507+0000 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Marking busy by address: [3813aa70-f51b-483f-a21e-42deeb54fdea port:33467, addresses:[localhost/127.0.0.1]]
2021-08-18T18:00:30.507+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on daemon addresses registry.
2021-08-18T18:00:30.507+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2021-08-18T18:00:30.508+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2021-08-18T18:00:30.508+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] resetting idle timer
2021-08-18T18:00:30.508+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running. Sleeping until state changes.
2021-08-18T18:00:30.508+0000 [INFO] [org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is about to start building Build{id=f70ac48f-808a-4631-b8f1-5812b9d97449, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src}.> Dispatching build started information...
2021-08-18T18:00:30.508+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 27: dispatching org.gradle.launcher.daemon.protocol.BuildStarted@1b96108
2021-08-18T18:00:30.509+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring env variables: [PATH, RUN_DISPLAY_URL, HUDSON_HOME, RUN_CHANGES_DISPLAY_URL, JOB_URL, HUDSON_COOKIE, MAIL, JENKINS_SERVER_COOKIE, LOGNAME, PWD, RUN_TESTS_DISPLAY_URL, JENKINS_URL, SHELL, BUILD_TAG, ROOT_BUILD_CAUSE, BUILD_CAUSE_TIMERTRIGGER, OLDPWD, GIT_CHECKOUT_DIR, JENKINS_HOME, sha1, CODECOV_TOKEN, NODE_NAME, BUILD_DISPLAY_NAME, JOB_DISPLAY_URL, GIT_BRANCH, SETUPTOOLS_USE_DISTUTILS, SHLVL, WORKSPACE_TMP, GIT_PREVIOUS_COMMIT, JAVA_HOME, BUILD_ID, LANG, XDG_SESSION_ID, JOB_NAME, SPARK_LOCAL_IP, BUILD_CAUSE, GIT_PREVIOUS_SUCCESSFUL_COMMIT, NODE_LABELS, HUDSON_URL, WORKSPACE, ROOT_BUILD_CAUSE_TIMERTRIGGER, _, GIT_COMMIT, COVERALLS_REPO_TOKEN, EXECUTOR_NUMBER, HUDSON_SERVER_COOKIE, SSH_CLIENT, JOB_BASE_NAME, USER, SSH_CONNECTION, BUILD_NUMBER, BUILD_URL, RUN_ARTIFACTS_DISPLAY_URL, GIT_URL, XDG_RUNTIME_DIR, HOME]
2021-08-18T18:00:30.510+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying all logs to the client via the connection.
2021-08-18T18:00:30.510+0000 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Removing daemon address: [3813aa70-f51b-483f-a21e-42deeb54fdea port:33467, addresses:[localhost/127.0.0.1]]
2021-08-18T18:00:30.510+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogToClient] The client will now receive all logging from the daemon (pid: 26320). The daemon log file: /home/jenkins/.gradle/daemon/6.8.3/daemon-26320.out.log
Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt.
2021-08-18T18:00:30.512+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on daemon addresses registry.
2021-08-18T18:00:30.512+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2021-08-18T18:00:30.513+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1571
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1571/display/redirect>
Changes:
------------------------------------------
[...truncated 22.52 MB...]
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629289334
nanos: 816410541
}
message: "Renamed 1 shards in 0.05 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629289334
nanos: 829130172
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629289334
nanos: 829305887
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629289334
nanos: 829392433
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:790"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629289334
nanos: 829483509
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629289334
nanos: 830399036
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629289334
nanos: 830546379
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 19.604936599731445 seconds.
INFO:root:Successfully completed job in 19.604936599731445 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35895
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7c51738e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7c51738ee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7c5173e670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempum829rl2/artifactsdmv_jusq' '--job-port' '48765' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:48765.
WARNING:root:Waiting for grpc channel to be ready at localhost:48765.
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38997'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:37397'
WARNING:root:Waiting for grpc channel to be ready at localhost:48765.
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:48765'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_2de10e41-795d-467b-8e6d-4b5c5b2f4854.'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_2de10e41-795d-467b-8e6d-4b5c5b2f4854.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_2de10e41-795d-467b-8e6d-4b5c5b2f4854.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:27 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_2de10e41-795d-467b-8e6d-4b5c5b2f4854.'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0818122227-92c40dc_6e3506bd-4fd8-4b07-bbe1-855ef43557b7'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:28 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0818122227-92c40dc_6e3506bd-4fd8-4b07-bbe1-855ef43557b7'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:29 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:31 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:36 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:36 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0818122227-92c40dc_6e3506bd-4fd8-4b07-bbe1-855ef43557b7 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40215.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33795.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:42887
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0818122227-92c40dc_6e3506bd-4fd8-4b07-bbe1-855ef43557b7: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/18 12:22:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0818122227-92c40dc_6e3506bd-4fd8-4b07-bbe1-855ef43557b7 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629289360.969417694","description":"Error received from peer ipv4:127.0.0.1:42887","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 659, in <lambda>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629289360.969475499","description":"Error received from peer ipv4:127.0.0.1:40215","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 642, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629289360.969445077","description":"Error received from peer ipv4:127.0.0.1:33795","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629289360.969417694","description":"Error received from peer ipv4:127.0.0.1:42887","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 25s
212 actionable tasks: 148 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ljb2uhanbj2zq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1570
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1570/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-5379] Go modules basic support and upgrade to Go 1.16.5.
[daniel.o.programmer] [BEAM-5379] Go Jenkins tests adjusted to use modules.
[daniel.o.programmer] [BEAM-5379] Add /v2/ to go module path and update references.
[daniel.o.programmer] [BEAM-5379] Fix Go SDK proto generation, and update protos.
[daniel.o.programmer] [BEAM-5379] Go module work fixup.
------------------------------------------
[...truncated 29.77 MB...]
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 5 failed, 61 passed, 8 skipped, 189 warnings in 6631.04 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 146
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 24s
212 actionable tasks: 150 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/npxff62iphtjc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1569
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1569/display/redirect?page=changes>
Changes:
[ruane.jb] Fix package containing RestrictionProvider
[randomstep] BEAM-12635 Bump Apache Compress to 1.21
[noreply] Fix line length
[noreply] Fix whitespace.
[noreply] Take formatters suggestion
[noreply] [BEAM-11088] Clean up incorrect comment (#15345)
[noreply] [BEAM-10955] Update Flink minor versions and enable testSavepointRest…
[noreply] Fix grpc data read thread block with finished instruction_id in
------------------------------------------
[...truncated 29.93 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 5 failed, 61 passed, 8 skipped, 183 warnings in 6705.33 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 15s
221 actionable tasks: 187 executed, 30 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/w4l4ztsvunlk2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1568
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1568/display/redirect?page=changes>
Changes:
[Jan Lukavský] [BEAM-12710] Disable TestStreamTest.testFirstElementLate for portable
[aromanenko.dev] [BEAM-12429] Add support for S3 Bucket Key at the object level
[noreply] [BEAM-12755] Stop throwing exceptions during formatting and calculation
[noreply] [BEAM-12734] Autopopulate opts using Go flags. (#15311)
[noreply] [BEAM-12768] Make the test less strict and instead match on substring
------------------------------------------
[...truncated 29.75 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 6 failed, 60 passed, 8 skipped, 187 warnings in 6735.03 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 27s
221 actionable tasks: 168 executed, 49 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/dakfpye3gfnkw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1567
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1567/display/redirect>
Changes:
------------------------------------------
[...truncated 7.11 MB...]
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629203204
nanos: 494966745
}
message: "Renamed 1 shards in 0.08 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629203204
nanos: 513347864
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629203204
nanos: 513563632
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629203204
nanos: 513642311
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629203204
nanos: 513718366
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629203204
nanos: 514760494
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629203204
nanos: 514919281
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 12.227488279342651 seconds.
INFO:root:Successfully completed job in 12.227488279342651 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:32827
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fbf89814e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fbf89814ee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fbf8981a670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp9qtcoca5/artifactsr7dbtaku' '--job-port' '59335' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:59335.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41135'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46413'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:59335'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_985d900e-cd78-4a3a-903c-946ee9906e68.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_985d900e-cd78-4a3a-903c-946ee9906e68.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_985d900e-cd78-4a3a-903c-946ee9906e68.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_985d900e-cd78-4a3a-903c-946ee9906e68.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:56 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0817122656-b922885b_f5742c90-4db9-4c6f-90e1-7e9d1cf1ea8b'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:56 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0817122656-b922885b_f5742c90-4db9-4c6f-90e1-7e9d1cf1ea8b'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:57 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:26:59 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:02 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:02 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0817122656-b922885b_f5742c90-4db9-4c6f-90e1-7e9d1cf1ea8b on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45209.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34669.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35319
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0817122656-b922885b_f5742c90-4db9-4c6f-90e1-7e9d1cf1ea8b: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.03 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 12:27:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0817122656-b922885b_f5742c90-4db9-4c6f-90e1-7e9d1cf1ea8b finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629203226.715758314","description":"Error received from peer ipv4:127.0.0.1:35319","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629203226.715797711","description":"Error received from peer ipv4:127.0.0.1:34669","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629203226.715803144","description":"Error received from peer ipv4:127.0.0.1:45209","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629203226.715758314","description":"Error received from peer ipv4:127.0.0.1:35319","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 57s
221 actionable tasks: 157 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7upfjlr4rlkzq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1566
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1566/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11088] TestStream utility and testing improvements (#15320)
------------------------------------------
[...truncated 258.52 KB...]
INFO:root:severity: INFO
timestamp {
seconds: 1629181799
nanos: 693310022
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629181799
nanos: 693485260
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629181799
nanos: 693564414
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629181799
nanos: 693632602
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629181799
nanos: 694816589
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629181799
nanos: 694967031
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 19.621397495269775 seconds.
INFO:root:Successfully completed job in 19.621397495269775 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35995
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8340103e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f8340103ee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8340109670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempzt0gcr6z/artifactsx6e96yqp' '--job-port' '57259' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:57259.
WARNING:root:Waiting for grpc channel to be ready at localhost:57259.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:08 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41659'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43335'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:57259'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:57259.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_30476002-2d53-4b6c-99cd-bd311ea62c68.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_30476002-2d53-4b6c-99cd-bd311ea62c68.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_30476002-2d53-4b6c-99cd-bd311ea62c68.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_30476002-2d53-4b6c-99cd-bd311ea62c68.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:11 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0817063011-9ade948b_d7feccd8-c4d4-405f-b1dd-413a2f904406'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:12 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0817063011-9ade948b_d7feccd8-c4d4-405f-b1dd-413a2f904406'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:13 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:15 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/08/17 06:30:18 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:19 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:19 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0817063011-9ade948b_d7feccd8-c4d4-405f-b1dd-413a2f904406 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33441.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:39943.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43961
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:22 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:22 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0817063011-9ade948b_d7feccd8-c4d4-405f-b1dd-413a2f904406: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.07 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 06:30:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0817063011-9ade948b_d7feccd8-c4d4-405f-b1dd-413a2f904406 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629181824.469060168","description":"Error received from peer ipv4:127.0.0.1:39943","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629181824.469109829","description":"Error received from peer ipv4:127.0.0.1:33441","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629181824.469057189","description":"Error received from peer ipv4:127.0.0.1:43961","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629181824.469057189","description":"Error received from peer ipv4:127.0.0.1:43961","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 146
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 37m 41s
221 actionable tasks: 157 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/k3e3ry6rw24va
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1565
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1565/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12609] Make type parameter part of PushdownProjector interface.
[Jeremy Quinn] Add AWS services as a runtime dependency to support S3
[pascal.gillet] [BEAM-12479] Fixes UnsupportedOperationException
[baeminbo] [BEAM-12504] Make CreateTransaction wait on input signal
[noreply] [BEAM-7745] Avoid uncached state fetches for streaming side-inputs
[Robert Burke] redundant build
------------------------------------------
[...truncated 1.18 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629160199
nanos: 142447948
}
message: "Renamed 1 shards in 0.08 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629160199
nanos: 151331186
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629160199
nanos: 151544094
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629160199
nanos: 151668310
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629160199
nanos: 151779651
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629160199
nanos: 152240037
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629160199
nanos: 152351379
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 18.995224714279175 seconds.
INFO:root:Successfully completed job in 18.995224714279175 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38033
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7c7897de50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7c7897dee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7c78982670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempfypdugz_/artifactsduclme62' '--job-port' '57233' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:57233.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33629'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39091'
WARNING:root:Waiting for grpc channel to be ready at localhost:57233.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:57233'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:57233.
WARNING:root:Waiting for grpc channel to be ready at localhost:57233.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_f5f82b54-65b7-497d-92fd-a8d9e793bf59.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_f5f82b54-65b7-497d-92fd-a8d9e793bf59.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_f5f82b54-65b7-497d-92fd-a8d9e793bf59.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_f5f82b54-65b7-497d-92fd-a8d9e793bf59.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:12 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0817003012-aa8df3d7_d9bf01a8-d904-4fb7-8948-e1b491230513'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:12 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0817003012-aa8df3d7_d9bf01a8-d904-4fb7-8948-e1b491230513'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:13 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:15 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:18 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:18 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0817003012-aa8df3d7_d9bf01a8-d904-4fb7-8948-e1b491230513 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41453.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44599.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36391
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:20 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0817003012-aa8df3d7_d9bf01a8-d904-4fb7-8948-e1b491230513: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.05 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/17 00:30:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0817003012-aa8df3d7_d9bf01a8-d904-4fb7-8948-e1b491230513 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629160222.518937833","description":"Error received from peer ipv4:127.0.0.1:41453","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629160222.518901907","description":"Error received from peer ipv4:127.0.0.1:36391","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629160222.518937889","description":"Error received from peer ipv4:127.0.0.1:44599","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629160222.518901907","description":"Error received from peer ipv4:127.0.0.1:36391","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 37s
221 actionable tasks: 160 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/s3b4ldbm23vdi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1564
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1564/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-12619] Swap LinkedBlockingQueue to ArrayBlockingQueue for minor
[Luke Cwik] fixup! Fix spotbugs warning
[Andrew Pilloud] [BEAM-12759] ORDER BY then SELECT
------------------------------------------
[...truncated 309.12 KB...]
INFO:root:severity: INFO
timestamp {
seconds: 1629139396
nanos: 744343757
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629139396
nanos: 764509439
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629139396
nanos: 764797449
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629139396
nanos: 764925241
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629139396
nanos: 765094757
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629139396
nanos: 766472101
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629139396
nanos: 766677856
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 29.23083257675171 seconds.
INFO:root:Successfully completed job in 29.23083257675171 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:43453
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f376b8d4e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f376b8d4ee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f376b8da670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temptzv8o8v6/artifactssk5wdjvq' '--job-port' '52949' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:52949.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:33 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39639'
WARNING:root:Waiting for grpc channel to be ready at localhost:52949.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42043'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:52949'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:52949.
WARNING:root:Waiting for grpc channel to be ready at localhost:52949.
WARNING:root:Waiting for grpc channel to be ready at localhost:52949.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d42b7dec-42d9-46ac-9161-e3f4fc8db61e.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d42b7dec-42d9-46ac-9161-e3f4fc8db61e.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d42b7dec-42d9-46ac-9161-e3f4fc8db61e.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d42b7dec-42d9-46ac-9161-e3f4fc8db61e.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:42 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0816184342-fdb9c91c_8849d914-3cad-42bd-8078-488c11c2e15b'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:42 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0816184342-fdb9c91c_8849d914-3cad-42bd-8078-488c11c2e15b'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:45 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/08/16 18:43:48 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:49 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:49 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:49 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0816184342-fdb9c91c_8849d914-3cad-42bd-8078-488c11c2e15b on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45599.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35765.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36883
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:51 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0816184342-fdb9c91c_8849d914-3cad-42bd-8078-488c11c2e15b: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.04 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 18:43:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0816184342-fdb9c91c_8849d914-3cad-42bd-8078-488c11c2e15b finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629139433.432567440","description":"Error received from peer ipv4:127.0.0.1:36883","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629139433.432643242","description":"Error received from peer ipv4:127.0.0.1:45599","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629139433.432567440","description":"Error received from peer ipv4:127.0.0.1:36883","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629139433.432605168","description":"Error received from peer ipv4:127.0.0.1:35765","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 43m 59s
217 actionable tasks: 157 executed, 56 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3e4mbaj5kbl5m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1563
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1563/display/redirect>
Changes:
------------------------------------------
[...truncated 11.34 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629116914
nanos: 777060270
}
message: "Renamed 1 shards in 0.03 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629116914
nanos: 786010742
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629116914
nanos: 786172628
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629116914
nanos: 786262273
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629116914
nanos: 786349058
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629116914
nanos: 786830902
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629116914
nanos: 786943674
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 9.36238694190979 seconds.
INFO:root:Successfully completed job in 9.36238694190979 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35483
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f724764ae50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f724764aee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7247650670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempssal42h4/artifactselr0_h74' '--job-port' '39387' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:39387.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38749'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46045'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39387'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:39387.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_62489f86-8301-490b-ba25-1f8f662527c3.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_62489f86-8301-490b-ba25-1f8f662527c3.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_62489f86-8301-490b-ba25-1f8f662527c3.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_62489f86-8301-490b-ba25-1f8f662527c3.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0816122844-b431cca6_9c5f1821-8487-4610-8718-9a81a01e1a3e'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0816122844-b431cca6_9c5f1821-8487-4610-8718-9a81a01e1a3e'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:45 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:46 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:50 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:50 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0816122844-b431cca6_9c5f1821-8487-4610-8718-9a81a01e1a3e on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36611.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42173.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:39629
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:52 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:52 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0816122844-b431cca6_9c5f1821-8487-4610-8718-9a81a01e1a3e: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.06 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 12:28:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0816122844-b431cca6_9c5f1821-8487-4610-8718-9a81a01e1a3e finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629116934.344602222","description":"Error received from peer ipv4:127.0.0.1:36611","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629116934.344577351","description":"Error received from peer ipv4:127.0.0.1:39629","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629116934.344585522","description":"Error received from peer ipv4:127.0.0.1:42173","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629116934.344577351","description":"Error received from peer ipv4:127.0.0.1:39629","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 39s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5fcz33xlkqjba
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1562
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1562/display/redirect>
Changes:
------------------------------------------
[...truncated 8.75 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629095163
nanos: 325548887
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629095163
nanos: 349916696
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629095163
nanos: 350109100
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629095163
nanos: 350208282
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629095163
nanos: 350301504
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629095163
nanos: 351249694
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629095163
nanos: 351392269
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 24.65223455429077 seconds.
INFO:root:Successfully completed job in 24.65223455429077 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:33773
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f656cc68e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f656cc68ee0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f656cc6e670> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempwkn__5up/artifactsjjia3t5d' '--job-port' '60485' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:60485.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37861'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:37833'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:13 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:60485'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:13 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:60485.
WARNING:root:Waiting for grpc channel to be ready at localhost:60485.
WARNING:root:Waiting for grpc channel to be ready at localhost:60485.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_2ec9c7f0-f110-4b3b-afd0-af075b7698f0.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_2ec9c7f0-f110-4b3b-afd0-af075b7698f0.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_2ec9c7f0-f110-4b3b-afd0-af075b7698f0.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_2ec9c7f0-f110-4b3b-afd0-af075b7698f0.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0816062617-7e3c6442_7f276e7e-58f0-4e7a-8522-e918f0a2e3e1'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:17 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0816062617-7e3c6442_7f276e7e-58f0-4e7a-8522-e918f0a2e3e1'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:19 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:23 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:23 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0816062617-7e3c6442_7f276e7e-58f0-4e7a-8522-e918f0a2e3e1 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:46747.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38649.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41015
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0816062617-7e3c6442_7f276e7e-58f0-4e7a-8522-e918f0a2e3e1: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.03 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/16 06:26:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0816062617-7e3c6442_7f276e7e-58f0-4e7a-8522-e918f0a2e3e1 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629095188.276121140","description":"Error received from peer ipv4:127.0.0.1:41015","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629095188.276159324","description":"Error received from peer ipv4:127.0.0.1:46747","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629095188.276121140","description":"Error received from peer ipv4:127.0.0.1:41015","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629095188.276148655","description":"Error received from peer ipv4:127.0.0.1:38649","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 59s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/t4ve4ufclmlm2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1561/display/redirect>
Changes:
------------------------------------------
[...truncated 10.72 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 4 failed, 62 passed, 8 skipped, 185 warnings in 5561.87 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 37m 19s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/s3kxiwqhmvdy6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1560/display/redirect>
Changes:
------------------------------------------
[...truncated 24.45 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 5 failed, 61 passed, 8 skipped, 187 warnings in 6654.84 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 26s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nzozlzodbamxk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1559/display/redirect>
Changes:
------------------------------------------
[...truncated 24.48 MB...]
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 5 failed, 61 passed, 8 skipped, 189 warnings in 6565.32 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 146
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 51s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6zwqlmm3wdhpq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1558/display/redirect>
Changes:
------------------------------------------
[...truncated 176.59 KB...]
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1629008670
nanos: 785457134
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629008670
nanos: 785623550
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629008670
nanos: 785694360
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629008670
nanos: 785767793
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629008670
nanos: 786631822
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1629008670
nanos: 786773920
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 26.071908950805664 seconds.
INFO:root:Successfully completed job in 26.071908950805664 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45289
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7efcb3f3e160> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7efcb3f3e1f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7efcb3f3e940> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempvjnfoi3d/artifactsaf2fmuml' '--job-port' '52835' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42507'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:38889'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:52835'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:52835.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_87421c67-5a4e-4c21-8791-ee4c8e4a177e.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_87421c67-5a4e-4c21-8791-ee4c8e4a177e.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_87421c67-5a4e-4c21-8791-ee4c8e4a177e.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_87421c67-5a4e-4c21-8791-ee4c8e4a177e.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:40 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0815062440-ecef837a_575d9b89-b79f-4f54-96b6-c0f228474afa'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:41 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0815062440-ecef837a_575d9b89-b79f-4f54-96b6-c0f228474afa'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:42 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:43 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:46 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:46 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0815062440-ecef837a_575d9b89-b79f-4f54-96b6-c0f228474afa on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40247.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41271.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38911
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0815062440-ecef837a_575d9b89-b79f-4f54-96b6-c0f228474afa: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.07 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/15 06:24:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0815062440-ecef837a_575d9b89-b79f-4f54-96b6-c0f228474afa finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629008691.276942031","description":"Error received from peer ipv4:127.0.0.1:38911","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629008691.276977081","description":"Error received from peer ipv4:127.0.0.1:40247","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1629008691.276965255","description":"Error received from peer ipv4:127.0.0.1:41271","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1629008691.276942031","description":"Error received from peer ipv4:127.0.0.1:38911","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 146
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 36s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/dunop3mjz24uu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1557
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1557/display/redirect>
Changes:
------------------------------------------
[...truncated 13.94 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628986941
nanos: 207134962
}
message: "Renamed 1 shards in 0.05 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628986941
nanos: 221160173
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628986941
nanos: 221402883
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628986941
nanos: 221527099
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628986941
nanos: 221638917
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628986941
nanos: 222609758
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628986941
nanos: 222752809
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 11.566556930541992 seconds.
INFO:root:Successfully completed job in 11.566556930541992 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:43751
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1db4a55160> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f1db4a551f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1db4a55940> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempurmrvyfn/artifacts_oydccrw' '--job-port' '33751' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:33751.
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33951'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35881'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33751'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:33751.
WARNING:root:Waiting for grpc channel to be ready at localhost:33751.
WARNING:root:Waiting for grpc channel to be ready at localhost:33751.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_f7002bcc-c9d4-44cf-aab1-f30e55dc36b5.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_f7002bcc-c9d4-44cf-aab1-f30e55dc36b5.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_f7002bcc-c9d4-44cf-aab1-f30e55dc36b5.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_f7002bcc-c9d4-44cf-aab1-f30e55dc36b5.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:38 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0815002237-7fe18eb7_692721a1-b6c7-4eab-a9db-8e82a5f40489'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:38 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0815002237-7fe18eb7_692721a1-b6c7-4eab-a9db-8e82a5f40489'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:38 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:41 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:44 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:44 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0815002237-7fe18eb7_692721a1-b6c7-4eab-a9db-8e82a5f40489 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45925.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35301.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33197
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:46 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0815002237-7fe18eb7_692721a1-b6c7-4eab-a9db-8e82a5f40489: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.20 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/15 00:22:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0815002237-7fe18eb7_692721a1-b6c7-4eab-a9db-8e82a5f40489 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628986968.650726892","description":"Error received from peer ipv4:127.0.0.1:33197","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628986968.650726892","description":"Error received from peer ipv4:127.0.0.1:33197","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628986968.650742112","description":"Error received from peer ipv4:127.0.0.1:35301","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628986968.650748781","description":"Error received from peer ipv4:127.0.0.1:45925","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 15s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fh6ti2p3rvacy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1556
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1556/display/redirect>
Changes:
------------------------------------------
[...truncated 174.28 KB...]
INFO:root:severity: INFO
timestamp {
seconds: 1628965726
nanos: 958381414
}
message: "Renamed 1 shards in 0.03 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-13"
INFO:root:severity: INFO
timestamp {
seconds: 1628965726
nanos: 966494321
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628965726
nanos: 966663599
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628965726
nanos: 966741800
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628965726
nanos: 966820240
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628965726
nanos: 968054533
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628965726
nanos: 968200683
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.894100666046143 seconds.
INFO:root:Successfully completed job in 7.894100666046143 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35349
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f17ff7cf160> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f17ff7cf1f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f17ff7cf940> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempiv5zau_l/artifactst98tivmb' '--job-port' '39593' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:39593.
WARNING:root:Waiting for grpc channel to be ready at localhost:39593.
WARNING:root:Waiting for grpc channel to be ready at localhost:39593.
WARNING:root:Waiting for grpc channel to be ready at localhost:39593.
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:28:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33963'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:28:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42169'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:28:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39593'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:28:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:39593.
WARNING:root:Waiting for grpc channel to be ready at localhost:39593.
WARNING:root:Waiting for grpc channel to be ready at localhost:39593.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_eb09e649-2ea4-49a2-b028-8b77b393d171.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_eb09e649-2ea4-49a2-b028-8b77b393d171.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_eb09e649-2ea4-49a2-b028-8b77b393d171.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_eb09e649-2ea4-49a2-b028-8b77b393d171.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0814182908-d16839cb_a0285bf9-5fd1-4401-a1f2-1992ebfd5047'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0814182908-d16839cb_a0285bf9-5fd1-4401-a1f2-1992ebfd5047'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:09 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:10 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:14 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:14 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0814182908-d16839cb_a0285bf9-5fd1-4401-a1f2-1992ebfd5047 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41245.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33783.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37197
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0814182908-d16839cb_a0285bf9-5fd1-4401-a1f2-1992ebfd5047: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.07 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/14 18:29:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0814182908-d16839cb_a0285bf9-5fd1-4401-a1f2-1992ebfd5047 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628965758.070153478","description":"Error received from peer ipv4:127.0.0.1:37197","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628965758.070205916","description":"Error received from peer ipv4:127.0.0.1:41245","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628965758.070153478","description":"Error received from peer ipv4:127.0.0.1:37197","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628965758.070153816","description":"Error received from peer ipv4:127.0.0.1:33783","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 35m 57s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/rnsaarhbjhd6m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1555
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1555/display/redirect>
Changes:
------------------------------------------
[...truncated 24.51 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 5 failed, 61 passed, 8 skipped, 193 warnings in 6633.84 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 5s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xvxdtiual4rh2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1554
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1554/display/redirect>
Changes:
------------------------------------------
[...truncated 256.30 KB...]
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628922372
nanos: 237379312
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628922372
nanos: 237539052
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628922372
nanos: 237610101
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628922372
nanos: 237675428
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628922372
nanos: 238199949
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628922372
nanos: 238331556
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 22.50228476524353 seconds.
INFO:root:Successfully completed job in 22.50228476524353 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42413
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2e69468160> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f2e694681f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2e69468940> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempgtbappm_/artifactsfw3vmul4' '--job-port' '33401' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38259'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:38183'
WARNING:root:Waiting for grpc channel to be ready at localhost:33401.
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33401'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_b7a063dc-fb01-4cad-928d-a4f6e843b66b.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_b7a063dc-fb01-4cad-928d-a4f6e843b66b.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_b7a063dc-fb01-4cad-928d-a4f6e843b66b.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_b7a063dc-fb01-4cad-928d-a4f6e843b66b.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:20 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0814062620-45f44df5_d91930af-a7f8-4d6b-bfc1-a1183797c635'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:20 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0814062620-45f44df5_d91930af-a7f8-4d6b-bfc1-a1183797c635'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:21 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:23 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:26 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:26 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0814062620-45f44df5_d91930af-a7f8-4d6b-bfc1-a1183797c635 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35033.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43099.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36289
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0814062620-45f44df5_d91930af-a7f8-4d6b-bfc1-a1183797c635: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.05 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/14 06:26:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0814062620-45f44df5_d91930af-a7f8-4d6b-bfc1-a1183797c635 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628922391.256180514","description":"Error received from peer ipv4:127.0.0.1:35033","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628922391.256180662","description":"Error received from peer ipv4:127.0.0.1:36289","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628922391.256180662","description":"Error received from peer ipv4:127.0.0.1:36289","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628922391.256179520","description":"Error received from peer ipv4:127.0.0.1:43099","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 146
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 54s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/tt5chbmfglxqg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1553
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1553/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12757]: Remove call for Json Dag in Portable PipelineRunner as
[noreply] fix confusing function def (#15120)
------------------------------------------
[...truncated 24.50 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 5 failed, 61 passed, 8 skipped, 183 warnings in 5712.29 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 17s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ols6nbfdzef4u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python38 - Build # 1552 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python38 - Build # 1552 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python38/1552/ to view the results.
beam_PostCommit_Python38 - Build # 1551 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python38 - Build # 1551 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python38/1551/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python38 #1550
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1550/display/redirect>
Changes:
------------------------------------------
[...truncated 8.81 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628835981
nanos: 820477724
}
message: "Renamed 1 shards in 0.04 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628835981
nanos: 826233386
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628835981
nanos: 826403379
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628835981
nanos: 826483964
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628835981
nanos: 826553583
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628835981
nanos: 827692985
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628835981
nanos: 827829837
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 10.528498411178589 seconds.
INFO:root:Successfully completed job in 10.528498411178589 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35123
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7eff387fc160> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7eff387fc1f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7eff387fc940> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temph4sppweg/artifacts_h4neirs' '--job-port' '33241' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:33241.
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:44345'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42909'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33241'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:33241.
WARNING:root:Waiting for grpc channel to be ready at localhost:33241.
WARNING:root:Waiting for grpc channel to be ready at localhost:33241.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1dff5a6c-b32f-48f0-aa05-3313b4970d28.'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1dff5a6c-b32f-48f0-aa05-3313b4970d28.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_1dff5a6c-b32f-48f0-aa05-3313b4970d28.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1dff5a6c-b32f-48f0-aa05-3313b4970d28.'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:36 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0813062636-5ccc610a_2ac049cc-cef4-48ec-96d4-150f655c7a37'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:36 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0813062636-5ccc610a_2ac049cc-cef4-48ec-96d4-150f655c7a37'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:37 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:39 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:43 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:43 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0813062636-5ccc610a_2ac049cc-cef4-48ec-96d4-150f655c7a37 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39331.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44583.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35173
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:46 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0813062636-5ccc610a_2ac049cc-cef4-48ec-96d4-150f655c7a37: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/13 06:26:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0813062636-5ccc610a_2ac049cc-cef4-48ec-96d4-150f655c7a37 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628836008.202899445","description":"Error received from peer ipv4:127.0.0.1:35173","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628836008.202927998","description":"Error received from peer ipv4:127.0.0.1:39331","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628836008.202928527","description":"Error received from peer ipv4:127.0.0.1:44583","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628836008.202899445","description":"Error received from peer ipv4:127.0.0.1:35173","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 2s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jsogcgnhbu7da
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1549
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1549/display/redirect?page=changes>
Changes:
[noreply] Loosen typing extensions requirement
[Kyle Weaver] [BEAM-12719] ZetaSQL docs: replace out of date table with link to
[ryanthompson591] Add retry_gcs_file_copy when 500 errros happen from server
[noreply] Merge pull request #15265 from [BEAM-12545] Extend FileSink by
[noreply] [BEAM-12453]: Add interface to access I/O topic information for a Samza
[noreply] [BEAM-10212] Guard state caching with experiment (#15319)
------------------------------------------
[...truncated 24.61 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
apache_beam/io/gcp/bigquery.py:1929
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1929: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
apache_beam/io/gcp/bigquery.py:1931
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1931: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
apache_beam/io/gcp/bigquery.py:1955
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1955: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
pipeline_options=pcoll.pipeline.options,
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
apache_beam/io/gcp/bigquery.py:1684
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1684: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/avro/schema.py>:1249: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
warnings.warn("`Parse` is deprecated in avro 1.9.2. "
apache_beam/io/gcp/bigquery_test.py:1137
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1137: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:178
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:178: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:165
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:165: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(
apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
io.BigQuerySink(
apache_beam/ml/gcp/cloud_dlp_it_test.py:74
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
| MaskDetectedDetails(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:159
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:159: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations_update) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:120
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:120: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/ml/gcp/cloud_dlp_it_test.py:85
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
| InspectForDetails(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:122
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:122: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
p | beam.Create(mutations) | WriteToSpanner(
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:108
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:108: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
r = p | ReadFromSpanner(
apache_beam/io/gcp/bigquery_read_it_test.py:281
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:281: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:395
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:395: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2061
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2061: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2062
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2062: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2075
<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2075: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
[31m[1m======= 6 failed, 60 passed, 8 skipped, 183 warnings in 5668.96 seconds ========[0m
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 37m 27s
217 actionable tasks: 164 executed, 49 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ani3ka3lpwf3y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1548
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1548/display/redirect?page=changes>
Changes:
[bsindhwani] Python Kata - Event Time Triggers
[bsindhwani] Python Kata - Event Time Triggers - Added license to task-info.yaml
[bsindhwani] Python Kata - Event Time Triggers - refactored the accumulation mode
[bsindhwani] Python Kata - Event Time Triggers - task description corrected to
[bsindhwani] Python Kata - Event Time Triggers - added streaming options
[bsindhwani] Python Kata - Event Time Triggers - add license text in task-info.yaml
[bsindhwani] Python Kata - Early Triggers
[bsindhwani] Python Kata - Event Time Triggers & Early Triggers - task.md - corrected
[bsindhwani] Python Kata - Window Accumulation Mode
[bsindhwani] Python Kata - Event Time Triggers - removed the external transform
[bsindhwani] Python Kata - added license to course-info.yaml
[Robert Burke] [BEAM-11779] Remove appliance restriction.
[noreply] [BEAM-6374] Emit PCollection metrics from GoSDK (#15289)
[noreply] Merge pull request #15165 from [BEAM-12593] Verify DataFrame API on
------------------------------------------
[...truncated 14.14 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-11"
INFO:root:severity: INFO
timestamp {
seconds: 1628792770
nanos: 877621173
}
message: "Renamed 1 shards in 0.05 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-11"
INFO:root:severity: INFO
timestamp {
seconds: 1628792770
nanos: 897075414
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628792770
nanos: 897320985
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628792770
nanos: 897459506
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628792770
nanos: 897580385
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628792770
nanos: 898624658
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628792770
nanos: 898782968
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.197901248931885 seconds.
INFO:root:Successfully completed job in 8.197901248931885 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37021
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f69e35ce4c0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f69e35ce550> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f69e35ceca0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempv31xg33r/artifactswp1n2r_9' '--job-port' '50093' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:50093.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42661'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46299'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50093'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:50093.
WARNING:root:Waiting for grpc channel to be ready at localhost:50093.
WARNING:root:Waiting for grpc channel to be ready at localhost:50093.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_2898057d-0f2c-4e09-aae8-98f5cf9ed066.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_2898057d-0f2c-4e09-aae8-98f5cf9ed066.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_2898057d-0f2c-4e09-aae8-98f5cf9ed066.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_2898057d-0f2c-4e09-aae8-98f5cf9ed066.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:26 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0812182626-6bb3ae0a_f91baa20-bc8e-4929-bb78-05b388c164c5'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:26 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0812182626-6bb3ae0a_f91baa20-bc8e-4929-bb78-05b388c164c5'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:27 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:28 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:30 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0812182626-6bb3ae0a_f91baa20-bc8e-4929-bb78-05b388c164c5 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35779.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:39313.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40901
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:33 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:34 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812182626-6bb3ae0a_f91baa20-bc8e-4929-bb78-05b388c164c5: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 18:26:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812182626-6bb3ae0a_f91baa20-bc8e-4929-bb78-05b388c164c5 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628792797.095503006","description":"Error received from peer ipv4:127.0.0.1:35779","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628792797.095264052","description":"Error received from peer ipv4:127.0.0.1:39313","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628792797.095250138","description":"Error received from peer ipv4:127.0.0.1:40901","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628792797.095250138","description":"Error received from peer ipv4:127.0.0.1:40901","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 26s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4rui6t5oruvz2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1547/display/redirect>
Changes:
------------------------------------------
[...truncated 6.03 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628771255
nanos: 67314386
}
message: "Renamed 1 shards in 0.07 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628771255
nanos: 78257083
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628771255
nanos: 78480482
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628771255
nanos: 78610658
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628771255
nanos: 78724622
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628771255
nanos: 80114364
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628771255
nanos: 80277681
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 26.702566146850586 seconds.
INFO:root:Successfully completed job in 26.702566146850586 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35187
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff262b66dc0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff262b66e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff262b6b5e0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temprzpy1zfk/artifacts388muf5s' '--job-port' '54793' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:54793.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:46 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43057'
WARNING:root:Waiting for grpc channel to be ready at localhost:54793.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:46 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:37453'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:46 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:54793'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:46 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:54793.
WARNING:root:Waiting for grpc channel to be ready at localhost:54793.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_48e11ef0-a9da-4b38-bc33-34653dc2c2bb.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_48e11ef0-a9da-4b38-bc33-34653dc2c2bb.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_48e11ef0-a9da-4b38-bc33-34653dc2c2bb.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_48e11ef0-a9da-4b38-bc33-34653dc2c2bb.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:52 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0812122752-7163af72_40b04c06-4d43-43a8-abc8-910be898c985'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:53 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0812122752-7163af72_40b04c06-4d43-43a8-abc8-910be898c985'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:53 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:55 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:58 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:58 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:27:58 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0812122752-7163af72_40b04c06-4d43-43a8-abc8-910be898c985 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45453.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40599.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45979
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:00 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812122752-7163af72_40b04c06-4d43-43a8-abc8-910be898c985: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.05 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 12:28:01 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812122752-7163af72_40b04c06-4d43-43a8-abc8-910be898c985 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628771282.654518835","description":"Error received from peer ipv4:127.0.0.1:45979","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628771282.654553382","description":"Error received from peer ipv4:127.0.0.1:45453","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628771282.654520776","description":"Error received from peer ipv4:127.0.0.1:40599","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628771282.654518835","description":"Error received from peer ipv4:127.0.0.1:45979","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 17s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/djf6uiay5n5yc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1546
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1546/display/redirect?page=changes>
Changes:
[noreply] [GoSDK Infra] Limit simultaneous tests binaries to 3. (#15321)
------------------------------------------
[...truncated 8.01 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628749656
nanos: 576357603
}
message: "Renamed 1 shards in 0.06 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628749656
nanos: 584231376
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628749656
nanos: 584415912
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628749656
nanos: 584492444
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628749656
nanos: 584566593
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628749656
nanos: 585896730
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628749656
nanos: 586081504
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 19.71204113960266 seconds.
INFO:root:Successfully completed job in 19.71204113960266 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45083
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff99625bdc0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff99625be50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff9962605e0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp1q3d1pdi/artifactsyqzgobr3' '--job-port' '50821' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:50821.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:44 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34957'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:45 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40173'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:45 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50821'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:45 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:50821.
WARNING:root:Waiting for grpc channel to be ready at localhost:50821.
WARNING:root:Waiting for grpc channel to be ready at localhost:50821.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:49 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_098bb042-73a1-4b50-a12c-bb644e696409.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:49 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_098bb042-73a1-4b50-a12c-bb644e696409.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:49 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_098bb042-73a1-4b50-a12c-bb644e696409.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:49 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_098bb042-73a1-4b50-a12c-bb644e696409.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:50 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0812062750-977050a7_500d33ff-295d-4d21-a3d7-23420fbb5e33'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:50 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0812062750-977050a7_500d33ff-295d-4d21-a3d7-23420fbb5e33'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:51 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:53 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:56 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:56 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0812062750-977050a7_500d33ff-295d-4d21-a3d7-23420fbb5e33 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42095.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43407.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35933
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:59 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:59 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:27:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812062750-977050a7_500d33ff-295d-4d21-a3d7-23420fbb5e33: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.07 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 06:28:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812062750-977050a7_500d33ff-295d-4d21-a3d7-23420fbb5e33 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628749681.560901179","description":"Error received from peer ipv4:127.0.0.1:35933","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628749681.560952145","description":"Error received from peer ipv4:127.0.0.1:42095","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628749681.560946393","description":"Error received from peer ipv4:127.0.0.1:43407","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628749681.560901179","description":"Error received from peer ipv4:127.0.0.1:35933","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 42m 40s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/epwxn5dqk3zgc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1545
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1545/display/redirect?page=changes>
Changes:
[marwant] Move misplaced cloud healthcare update info line from template to
[Robert Burke] [BEAM-12738][GoSDK] Fix Dataflow Job URL
[eugene.nikolayev] Fix docs typos in contrib guide.
[noreply] Merge pull request #15126 from [BEAM-12665] Add option in ReadAll
[noreply] Update ptransform.py error message (#15286)
------------------------------------------
[...truncated 572.13 KB...]
nanos: 388223409
}
message: "Renamed 1 shards in 0.03 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628727410
nanos: 397325992
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628727410
nanos: 397558689
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628727410
nanos: 397679328
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628727410
nanos: 397788763
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628727410
nanos: 399427890
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628727410
nanos: 399607181
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.0675787925720215 seconds.
INFO:root:Successfully completed job in 7.0675787925720215 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42781
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7afc5a0dc0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7afc5a0e50> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7afc5a55e0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempeiyc01na/artifactssi3__wqg' '--job-port' '37893' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39027'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:57 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34591'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:57 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:37893'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:57 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:37893.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a01ff0e5-df5e-4fd3-819d-7ce5c1c344bd.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a01ff0e5-df5e-4fd3-819d-7ce5c1c344bd.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a01ff0e5-df5e-4fd3-819d-7ce5c1c344bd.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:16:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a01ff0e5-df5e-4fd3-819d-7ce5c1c344bd.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:00 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0812001700-f0a2f4dd_3829ae00-a9bd-4dc7-b4b8-ece6ed0385b8'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:00 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0812001700-f0a2f4dd_3829ae00-a9bd-4dc7-b4b8-ece6ed0385b8'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:01 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:01 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:02 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:02 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0812001700-f0a2f4dd_3829ae00-a9bd-4dc7-b4b8-ece6ed0385b8 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45695.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37465.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44093
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:04 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812001700-f0a2f4dd_3829ae00-a9bd-4dc7-b4b8-ece6ed0385b8: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.04 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/12 00:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0812001700-f0a2f4dd_3829ae00-a9bd-4dc7-b4b8-ece6ed0385b8 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628727426.904137860","description":"Error received from peer ipv4:127.0.0.1:44093","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628727426.904137860","description":"Error received from peer ipv4:127.0.0.1:44093","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628727426.904885455","description":"Error received from peer ipv4:127.0.0.1:45695","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628727426.904167196","description":"Error received from peer ipv4:127.0.0.1:37465","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:container:pullLicenses'.
> Process 'command './license_scripts/license_script.sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 43m 47s
214 actionable tasks: 151 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vqvcbficl4kq6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1544
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1544/display/redirect>
Changes:
------------------------------------------
[...truncated 7.20 MB...]
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:297"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628706234
nanos: 264348983
}
message: "Renamed 1 shards in 0.01 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628706234
nanos: 271296024
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628706234
nanos: 271462917
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628706234
nanos: 271539211
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628706234
nanos: 271607637
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628706234
nanos: 272063255
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628706234
nanos: 272180557
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.34820032119751 seconds.
INFO:root:Successfully completed job in 5.34820032119751 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34643
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f34d3668550> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f34d36685e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f34d3668d30> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempt3j5bjro/artifactsw3ye6ksv' '--job-port' '54501' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:40449'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41805'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:54501'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9d7597fb-aced-4cdf-8483-3088d9c86e9c.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9d7597fb-aced-4cdf-8483-3088d9c86e9c.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_9d7597fb-aced-4cdf-8483-3088d9c86e9c.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:23:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9d7597fb-aced-4cdf-8483-3088d9c86e9c.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:00 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0811182400-845d2a9_6891a11a-b739-4eff-896f-cc4098b13313'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:00 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0811182400-845d2a9_6891a11a-b739-4eff-896f-cc4098b13313'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:00 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:01 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/08/11 18:24:02 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:02 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:02 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0811182400-845d2a9_6891a11a-b739-4eff-896f-cc4098b13313 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43561.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40829.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34325
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0811182400-845d2a9_6891a11a-b739-4eff-896f-cc4098b13313: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/11 18:24:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0811182400-845d2a9_6891a11a-b739-4eff-896f-cc4098b13313 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1628706245.582313477","description":"Error received from peer ipv4:127.0.0.1:43561","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628706245.582285979","description":"Error received from peer ipv4:127.0.0.1:34325","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628706245.582273755","description":"Error received from peer ipv4:127.0.0.1:40829","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628706245.582285979","description":"Error received from peer ipv4:127.0.0.1:34325","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 44m 46s
217 actionable tasks: 156 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fldfy7ml6bcz2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python38 #1543
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/1543/display/redirect>
Changes:
------------------------------------------
[...truncated 1.58 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/io/filebasedsink.py:345"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1628684609
nanos: 301357507
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628684609
nanos: 301581382
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628684609
nanos: 301722764
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:717"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628684609
nanos: 301824569
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:897"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628684609
nanos: 302303314
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1628684609
nanos: 302409172
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 9.529503107070923 seconds.
INFO:root:Successfully completed job in 9.529503107070923 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45157
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f61a6db1550> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f61a6db15e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f61a6db1d30> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.33.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempm84hmafe/artifactshhetss_6' '--job-port' '41777' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:33 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37713'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:33 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:45611'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:33 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:41777'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:33 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_77e5c788-6986-4652-88e0-f2b4b6de62a3.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_77e5c788-6986-4652-88e0-f2b4b6de62a3.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_77e5c788-6986-4652-88e0-f2b4b6de62a3.null.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_77e5c788-6986-4652-88e0-f2b4b6de62a3.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:35 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0811122335-48215edd_5eeaa069-51e9-40b9-9794-cb19bf4e1b4a'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:35 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0811122335-48215edd_5eeaa069-51e9-40b9-9794-cb19bf4e1b4a'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:35 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:36 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:36 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:36 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0811122335-48215edd_5eeaa069-51e9-40b9-9794-cb19bf4e1b4a on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40831.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37045.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:39031
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0811122335-48215edd_5eeaa069-51e9-40b9-9794-cb19bf4e1b4a: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/08/11 12:23:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0811122335-48215edd_5eeaa069-51e9-40b9-9794-cb19bf4e1b4a finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1028, in pull_responses
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628684619.508643193","description":"Error received from peer ipv4:127.0.0.1:39031","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628684619.508643167","description":"Error received from peer ipv4:127.0.0.1:37045","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 587, in <lambda>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628684619.508653181","description":"Error received from peer ipv4:127.0.0.1:40831","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 570, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1628684619.508643193","description":"Error received from peer ipv4:127.0.0.1:39031","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py38:postCommitPy38
> Task :sdks:python:test-suites:dataflow:py38:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 146
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 54
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 122
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 11s
217 actionable tasks: 154 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6j4ugljvchiyc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org