You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/05/20 10:13:59 UTC

Build failed in Jenkins: beam_PostCommit_Python38 #2661

See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2661/display/redirect?page=changes>

Changes:

[noreply] Remove unused validation configurations. (#17705)


------------------------------------------
[...truncated 35.36 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0520 08:29:38.598350155 3029113 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653035378
  nanos: 598088264
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653035378
  nanos: 598249912
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653035378
  nanos: 598326206
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653035378
  nanos: 598387718
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653035378
  nanos: 598580837
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653035378
  nanos: 598648309
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 21.921300411224365 seconds with state DONE.
INFO:root:Completed job in 21.921300411224365 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40767
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f75470ce1f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f75470ce280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f75470ce9d0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp9b7jbff5/artifactsb3ljalch' '--job-port' '44021' '--artifact-port' '0' '--expansion-port' '0']
E0520 08:29:43.949187817 3029390 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:45 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:45 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38243'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:45 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40385'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:46 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:44021'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:46 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:46 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_e99dcd00-9093-4bf8-bd03-53fe5cad8c68.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:46 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_e99dcd00-9093-4bf8-bd03-53fe5cad8c68.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:46 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_e99dcd00-9093-4bf8-bd03-53fe5cad8c68.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:46 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_e99dcd00-9093-4bf8-bd03-53fe5cad8c68.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:47 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0520082947-4b9ae2c8_3c5af72c-b1ed-414b-9601-caa0f29d692a'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:47 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0520082947-4b9ae2c8_3c5af72c-b1ed-414b-9601-caa0f29d692a'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:47 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:47 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:48 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:48 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:48 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0520082947-4b9ae2c8_3c5af72c-b1ed-414b-9601-caa0f29d692a on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42041.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34081.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33093
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0520082947-4b9ae2c8_3c5af72c-b1ed-414b-9601-caa0f29d692a: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/20 08:29:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0520082947-4b9ae2c8_3c5af72c-b1ed-414b-9601-caa0f29d692a finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653035391.137949833","description":"Error received from peer ipv6:[::1]:34081","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653035391.137956049","description":"Error received from peer ipv6:[::1]:42041","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653035391.137956047","description":"Error received from peer ipv6:[::1]:33093","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653035391.137956047","description":"Error received from peer ipv6:[::1]:33093","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 58m 34s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/fj5b7d3277m4y

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Jenkins build is back to normal : beam_PostCommit_Python38 #2679

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2679/display/redirect?page=changes>


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2678

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2678/display/redirect>

Changes:


------------------------------------------
[...truncated 44.20 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0524 14:28:33.317467642 1894128 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653402513
  nanos: 316958427
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653402513
  nanos: 317204952
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653402513
  nanos: 317345142
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653402513
  nanos: 317452192
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653402513
  nanos: 317728042
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653402513
  nanos: 317847728
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 9.998042106628418 seconds with state DONE.
INFO:root:Completed job in 9.998042106628418 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44133
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fead151e280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fead151e310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fead151ea60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempkedx0ha9/artifactsjr542kxv' '--job-port' '34977' '--artifact-port' '0' '--expansion-port' '0']
E0524 14:28:36.258146163 1895008 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:37 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36019'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:38739'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:34977'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:38 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:39 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_4cab086b-8d79-484d-8c58-b43a2d424e64.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:39 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_4cab086b-8d79-484d-8c58-b43a2d424e64.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:39 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_4cab086b-8d79-484d-8c58-b43a2d424e64.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:39 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_4cab086b-8d79-484d-8c58-b43a2d424e64.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:39 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0524142839-cc660f11_8d32e534-8b31-4c9c-862c-e4f612f6ff6d'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:39 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0524142839-cc660f11_8d32e534-8b31-4c9c-862c-e4f612f6ff6d'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:40 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:40 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0524142839-cc660f11_8d32e534-8b31-4c9c-862c-e4f612f6ff6d on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36539.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34069.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46081
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0524142839-cc660f11_8d32e534-8b31-4c9c-862c-e4f612f6ff6d: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/24 14:28:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0524142839-cc660f11_8d32e534-8b31-4c9c-862c-e4f612f6ff6d finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653402523.612887988","description":"Error received from peer ipv6:[::1]:36539","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653402523.612868558","description":"Error received from peer ipv6:[::1]:46081","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653402523.612869970","description":"Error received from peer ipv6:[::1]:34069","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653402523.612868558","description":"Error received from peer ipv6:[::1]:46081","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 55m 21s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/xh3l37e3n7nkm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2677

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2677/display/redirect?page=changes>

Changes:

[noreply] [BEAM-14489] Remove non-SDF version of TextIO. (#17712)


------------------------------------------
[...truncated 45.22 MB...]
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2204: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2519: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2521: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2545: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2194: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/io/gcp/tests/utils.py:100
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/dataframe/io.py:629
apache_beam/dataframe/io.py:629
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:629: FutureWarning: WriteToFiles is experimental.
    return pcoll | fileio.WriteToFiles(

apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

apache_beam/io/gcp/bigquery_test.py:1636
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1636: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    streaming = self.test_pipeline.options.view_as(StandardOptions).streaming

apache_beam/io/gcp/bigquery_read_it_test.py:169
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:169: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2563: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2584: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(

apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
    io.BigQuerySink(

apache_beam/ml/gcp/cloud_dlp_it_test.py:74
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
    | MaskDetectedDetails(

apache_beam/ml/gcp/cloud_dlp_it_test.py:85
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
    | InspectForDetails(

apache_beam/io/gcp/bigquery_read_it_test.py:566
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:566: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery_read_it_test.py:680
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:680: FutureWarning: ReadAllFromBigQuery is experimental.
    | beam.io.ReadAllFromBigQuery())

apache_beam/io/gcp/bigquery.py:2693
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2693: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2694
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2694: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2707
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2707: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    options=pcoll.pipeline.options,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
============ 76 passed, 8 skipped, 207 warnings in 6737.02 seconds =============

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 54m 10s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/kdsyqmyj77ugs

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2676

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2676/display/redirect?page=changes>

Changes:

[noreply] [BEAM-14499] Step global, unbounded side input case back to warning

[noreply] [BEAM-14484] Step back unexpected primary handling to warnings (#17724)

[noreply] [BEAM-14486] Document pubsubio & fix its behavior. (#17709)


------------------------------------------
[...truncated 36.02 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0524 02:28:34.348684964 1227229 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653359314
  nanos: 348335504
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653359314
  nanos: 348660707
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653359314
  nanos: 348801136
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653359314
  nanos: 348883152
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653359314
  nanos: 349153518
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653359314
  nanos: 349237203
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 10.31566834449768 seconds with state DONE.
INFO:root:Completed job in 10.31566834449768 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37495
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f137668e280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f137668e310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f137668ea60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempr4eq2m4m/artifacts8ii7nq7k' '--job-port' '60459' '--artifact-port' '0' '--expansion-port' '0']
E0524 02:28:37.895455263 1227494 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:39 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42091'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35505'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:60459'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_16ee6df5-f249-466b-a458-f25e4fb3b692.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_16ee6df5-f249-466b-a458-f25e4fb3b692.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_16ee6df5-f249-466b-a458-f25e4fb3b692.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_16ee6df5-f249-466b-a458-f25e4fb3b692.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:41 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0524022840-b826b7e5_ad2198a0-8b7d-4eff-8b2c-ca503646d886'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:41 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0524022840-b826b7e5_ad2198a0-8b7d-4eff-8b2c-ca503646d886'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:41 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:41 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:42 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:42 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0524022840-b826b7e5_ad2198a0-8b7d-4eff-8b2c-ca503646d886 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38033.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37193.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44359
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:43 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0524022840-b826b7e5_ad2198a0-8b7d-4eff-8b2c-ca503646d886: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/24 02:28:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0524022840-b826b7e5_ad2198a0-8b7d-4eff-8b2c-ca503646d886 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653359325.626440175","description":"Error received from peer ipv6:[::1]:44359","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653359325.626414845","description":"Error received from peer ipv6:[::1]:37193","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653359325.626440828","description":"Error received from peer ipv6:[::1]:38033","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653359325.626440175","description":"Error received from peer ipv6:[::1]:44359","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 57m 57s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/3bdym5bvhoiya

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2675

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2675/display/redirect?page=changes>

Changes:

[noreply] [BEAM-14494] Tag rc dockre container with format ${RELEASE}rc${RC_NUM}

[noreply] [BEAM-11578] Fix TypeError in dataflow_metrics has 0 distribution sum


------------------------------------------
[...truncated 37.64 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0523 20:29:02.399326384  862327 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653337742
  nanos: 398710727
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653337742
  nanos: 398956298
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653337742
  nanos: 399059295
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653337742
  nanos: 399138927
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653337742
  nanos: 399409294
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653337742
  nanos: 399503707
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 10.437193155288696 seconds with state DONE.
INFO:root:Completed job in 10.437193155288696 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:33267
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f101b50f280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f101b50f310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f101b50fa60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempbs3nmvg5/artifacts72v8o4up' '--job-port' '49325' '--artifact-port' '0' '--expansion-port' '0']
E0523 20:29:05.556681146  862589 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:07 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38641'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43749'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:49325'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_717ec457-b301-476c-b872-ecb3d818061d.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_717ec457-b301-476c-b872-ecb3d818061d.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_717ec457-b301-476c-b872-ecb3d818061d.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_717ec457-b301-476c-b872-ecb3d818061d.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:09 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0523202909-38bc7c7d_3f33d29f-cd90-4888-8740-9bb020b15813'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:09 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0523202909-38bc7c7d_3f33d29f-cd90-4888-8740-9bb020b15813'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:09 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:09 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:10 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:10 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:10 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0523202909-38bc7c7d_3f33d29f-cd90-4888-8740-9bb020b15813 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:34247.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43783.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44887
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0523202909-38bc7c7d_3f33d29f-cd90-4888-8740-9bb020b15813: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/23 20:29:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0523202909-38bc7c7d_3f33d29f-cd90-4888-8740-9bb020b15813 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653337753.807583494","description":"Error received from peer ipv6:[::1]:44887","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653337753.807583086","description":"Error received from peer ipv6:[::1]:43783","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653337753.807607364","description":"Error received from peer ipv6:[::1]:34247","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653337753.807583494","description":"Error received from peer ipv6:[::1]:44887","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 57m 40s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/4772r3ud5taes

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2674

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2674/display/redirect?page=changes>

Changes:

[noreply] Add clarification on Filter transform's input function to pydoc.

[noreply] [BEAM-14367]Flaky timeout in


------------------------------------------
[...truncated 30.13 MB...]
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2204: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2519: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2521: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2545: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2194: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/io/gcp/tests/utils.py:100
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/dataframe/io.py:629
apache_beam/dataframe/io.py:629
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:629: FutureWarning: WriteToFiles is experimental.
    return pcoll | fileio.WriteToFiles(

apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

apache_beam/io/gcp/bigquery_test.py:1636
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1636: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    streaming = self.test_pipeline.options.view_as(StandardOptions).streaming

apache_beam/io/gcp/bigquery_read_it_test.py:169
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:169: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2563: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2584: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(

apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
    io.BigQuerySink(

apache_beam/ml/gcp/cloud_dlp_it_test.py:74
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
    | MaskDetectedDetails(

apache_beam/ml/gcp/cloud_dlp_it_test.py:85
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
    | InspectForDetails(

apache_beam/io/gcp/bigquery_read_it_test.py:566
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:566: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery_read_it_test.py:680
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:680: FutureWarning: ReadAllFromBigQuery is experimental.
    | beam.io.ReadAllFromBigQuery())

apache_beam/io/gcp/bigquery.py:2693
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2693: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2694
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2694: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2707
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2707: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    options=pcoll.pipeline.options,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
============ 76 passed, 8 skipped, 203 warnings in 7185.81 seconds =============

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 2h 1m 57s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/lxobrzizjispm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2673

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2673/display/redirect>

Changes:


------------------------------------------
[...truncated 41.22 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0523 08:28:47.653170418  162739 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653294527
  nanos: 652810096
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653294527
  nanos: 653061866
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653294527
  nanos: 653149366
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653294527
  nanos: 653233766
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653294527
  nanos: 653581142
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653294527
  nanos: 653690338
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 10.286847114562988 seconds with state DONE.
INFO:root:Completed job in 10.286847114562988 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38571
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6c79e36280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f6c79e36310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6c79e36a60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temphbd0a3h7/artifacts2ut9oy74' '--job-port' '45033' '--artifact-port' '0' '--expansion-port' '0']
E0523 08:28:50.799285449  163065 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:52 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:52 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35125'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44469'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:45033'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_bb5c1664-5831-48c6-bfa4-c45b3b8b896e.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_bb5c1664-5831-48c6-bfa4-c45b3b8b896e.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_bb5c1664-5831-48c6-bfa4-c45b3b8b896e.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_bb5c1664-5831-48c6-bfa4-c45b3b8b896e.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:53 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0523082853-a2241854_8cd23fb9-3872-4252-9ad8-779ac914fc17'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:54 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0523082853-a2241854_8cd23fb9-3872-4252-9ad8-779ac914fc17'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:54 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:54 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:55 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:55 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:55 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0523082853-a2241854_8cd23fb9-3872-4252-9ad8-779ac914fc17 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42253.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46705.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46747
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:57 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0523082853-a2241854_8cd23fb9-3872-4252-9ad8-779ac914fc17: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:58 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:58 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:58 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:58 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:58 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/23 08:28:58 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0523082853-a2241854_8cd23fb9-3872-4252-9ad8-779ac914fc17 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653294538.686664977","description":"Error received from peer ipv6:[::1]:42253","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653294538.686678257","description":"Error received from peer ipv6:[::1]:46747","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653294538.686660375","description":"Error received from peer ipv6:[::1]:46705","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653294538.686678257","description":"Error received from peer ipv6:[::1]:46747","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 55m 47s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/uvfvydps5nfrm

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2672

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2672/display/redirect>

Changes:


------------------------------------------
[...truncated 32.00 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0523 02:27:37.812338365 4001063 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653272857
  nanos: 811938762
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653272857
  nanos: 812125921
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653272857
  nanos: 812205076
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653272857
  nanos: 812271595
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653272857
  nanos: 812491416
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653272857
  nanos: 812577009
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 9.844641208648682 seconds with state DONE.
INFO:root:Completed job in 9.844641208648682 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40673
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8393f36280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f8393f36310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8393f36a60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempp4xynn4s/artifactsd0uuzzv7' '--job-port' '53673' '--artifact-port' '0' '--expansion-port' '0']
E0523 02:27:40.687210930 4001323 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43291'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43091'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:53673'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_32e443c6-b603-4bf8-9e28-f54abd17d94a.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_32e443c6-b603-4bf8-9e28-f54abd17d94a.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_32e443c6-b603-4bf8-9e28-f54abd17d94a.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_32e443c6-b603-4bf8-9e28-f54abd17d94a.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0523022743-70825f8c_5c1340ad-6533-4c3c-aa4e-3e2b1a89b100'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0523022743-70825f8c_5c1340ad-6533-4c3c-aa4e-3e2b1a89b100'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:44 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:45 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:45 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:45 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0523022743-70825f8c_5c1340ad-6533-4c3c-aa4e-3e2b1a89b100 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35011.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41329.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37127
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0523022743-70825f8c_5c1340ad-6533-4c3c-aa4e-3e2b1a89b100: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/23 02:27:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0523022743-70825f8c_5c1340ad-6533-4c3c-aa4e-3e2b1a89b100 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653272868.049018890","description":"Error received from peer ipv6:[::1]:41329","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653272868.049148844","description":"Error received from peer ipv6:[::1]:37127","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653272868.049083037","description":"Error received from peer ipv6:[::1]:35011","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653272868.049148844","description":"Error received from peer ipv6:[::1]:37127","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 59m 31s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/rqbvlssyfxfdk

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2671

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2671/display/redirect>

Changes:


------------------------------------------
[...truncated 20.68 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0522 20:28:23.364376325 3664116 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653251303
  nanos: 364140272
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653251303
  nanos: 364360809
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653251303
  nanos: 364450216
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653251303
  nanos: 364538431
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653251303
  nanos: 364782094
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653251303
  nanos: 364865541
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 9.955548763275146 seconds with state DONE.
INFO:root:Completed job in 9.955548763275146 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:41107
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f332b766280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f332b766310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f332b766a60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempfgh6702m/artifacts9jcdn6f9' '--job-port' '57829' '--artifact-port' '0' '--expansion-port' '0']
E0522 20:28:26.336154314 3664380 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:28 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35551'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42857'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:57829'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:28 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_45a8acdf-c2d6-4261-aae4-b712451bf731.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_45a8acdf-c2d6-4261-aae4-b712451bf731.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_45a8acdf-c2d6-4261-aae4-b712451bf731.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_45a8acdf-c2d6-4261-aae4-b712451bf731.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:29 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0522202829-49f2f243_22d62678-ef6c-47bb-bbcb-262938acbc74'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:29 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0522202829-49f2f243_22d62678-ef6c-47bb-bbcb-262938acbc74'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:30 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:30 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:31 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:31 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0522202829-49f2f243_22d62678-ef6c-47bb-bbcb-262938acbc74 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40767.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42601.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37619
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0522202829-49f2f243_22d62678-ef6c-47bb-bbcb-262938acbc74: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/22 20:28:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0522202829-49f2f243_22d62678-ef6c-47bb-bbcb-262938acbc74 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653251313.959260099","description":"Error received from peer ipv6:[::1]:40767","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653251313.959178211","description":"Error received from peer ipv6:[::1]:37619","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653251313.959247427","description":"Error received from peer ipv6:[::1]:42601","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653251313.959178211","description":"Error received from peer ipv6:[::1]:37619","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 55m 39s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/ffia7g7corfsc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2670

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2670/display/redirect>

Changes:


------------------------------------------
[...truncated 45.12 MB...]
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2204: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2519: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2521: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2545: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2194: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/io/gcp/tests/utils.py:100
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/dataframe/io.py:629
apache_beam/dataframe/io.py:629
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:629: FutureWarning: WriteToFiles is experimental.
    return pcoll | fileio.WriteToFiles(

apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

apache_beam/io/gcp/bigquery_test.py:1636
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1636: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    streaming = self.test_pipeline.options.view_as(StandardOptions).streaming

apache_beam/io/gcp/bigquery_read_it_test.py:169
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:169: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2563: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2584: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(

apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
    io.BigQuerySink(

apache_beam/ml/gcp/cloud_dlp_it_test.py:74
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
    | MaskDetectedDetails(

apache_beam/ml/gcp/cloud_dlp_it_test.py:85
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
    | InspectForDetails(

apache_beam/io/gcp/bigquery_read_it_test.py:566
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:566: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery_read_it_test.py:680
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:680: FutureWarning: ReadAllFromBigQuery is experimental.
    | beam.io.ReadAllFromBigQuery())

apache_beam/io/gcp/bigquery.py:2693
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2693: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2694
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2694: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2707
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2707: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    options=pcoll.pipeline.options,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
============ 76 passed, 8 skipped, 217 warnings in 6838.81 seconds =============

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 56m
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/wv7y4fdjtkh3a

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2669

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2669/display/redirect>

Changes:


------------------------------------------
[...truncated 35.51 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0522 08:28:24.221507382 2984687 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653208104
  nanos: 220979690
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653208104
  nanos: 221213102
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653208104
  nanos: 221320152
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653208104
  nanos: 221416234
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653208104
  nanos: 222833633
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653208104
  nanos: 223019599
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 9.921796560287476 seconds with state DONE.
INFO:root:Completed job in 9.921796560287476 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37671
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f803b97e280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f803b97e310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f803b97ea60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempdb64tme1/artifactsesweh22u' '--job-port' '58113' '--artifact-port' '0' '--expansion-port' '0']
E0522 08:28:27.228895906 2984986 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:28 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43489'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42395'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:58113'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_595a15bd-5ad1-42e5-9667-86cb7a6f694b.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_595a15bd-5ad1-42e5-9667-86cb7a6f694b.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:29 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_595a15bd-5ad1-42e5-9667-86cb7a6f694b.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_595a15bd-5ad1-42e5-9667-86cb7a6f694b.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:30 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0522082830-eb18cdde_fe4c565b-9459-4f95-ba86-98593fa8277b'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:30 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0522082830-eb18cdde_fe4c565b-9459-4f95-ba86-98593fa8277b'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:30 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:30 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:31 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:31 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0522082830-eb18cdde_fe4c565b-9459-4f95-ba86-98593fa8277b on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33797.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36049.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37785
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0522082830-eb18cdde_fe4c565b-9459-4f95-ba86-98593fa8277b: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/22 08:28:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0522082830-eb18cdde_fe4c565b-9459-4f95-ba86-98593fa8277b finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653208114.360722453","description":"Error received from peer ipv6:[::1]:33797","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653208114.360708843","description":"Error received from peer ipv6:[::1]:36049","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653208114.360706611","description":"Error received from peer ipv6:[::1]:37785","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653208114.360706611","description":"Error received from peer ipv6:[::1]:37785","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 56m 54s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/q7m5udewscf3e

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2668

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2668/display/redirect>

Changes:


------------------------------------------
[...truncated 45.21 MB...]
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2204: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2519: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2521: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2545: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2194: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/io/gcp/tests/utils.py:100
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/dataframe/io.py:629
apache_beam/dataframe/io.py:629
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:629: FutureWarning: WriteToFiles is experimental.
    return pcoll | fileio.WriteToFiles(

apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

apache_beam/io/gcp/bigquery_test.py:1636
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1636: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    streaming = self.test_pipeline.options.view_as(StandardOptions).streaming

apache_beam/io/gcp/bigquery_read_it_test.py:169
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:169: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/ml/gcp/cloud_dlp_it_test.py:74
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
    | MaskDetectedDetails(

apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2563: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2584: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/ml/gcp/cloud_dlp_it_test.py:85
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
    | InspectForDetails(

apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(

apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
    io.BigQuerySink(

apache_beam/io/gcp/bigquery_read_it_test.py:566
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:566: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery_read_it_test.py:680
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:680: FutureWarning: ReadAllFromBigQuery is experimental.
    | beam.io.ReadAllFromBigQuery())

apache_beam/io/gcp/bigquery.py:2693
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2693: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2694
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2694: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2707
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2707: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    options=pcoll.pipeline.options,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
============ 76 passed, 8 skipped, 205 warnings in 6794.74 seconds =============

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 55m 21s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/gmrao6iao3xlu

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2667

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2667/display/redirect>

Changes:


------------------------------------------
[...truncated 34.22 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0521 20:28:14.773305091 2305121 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653164894
  nanos: 772981166
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653164894
  nanos: 773150444
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653164894
  nanos: 773228883
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653164894
  nanos: 773296594
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653164894
  nanos: 773882150
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653164894
  nanos: 774022817
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 9.676787853240967 seconds with state DONE.
INFO:root:Completed job in 9.676787853240967 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40687
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f36a81c7280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f36a81c7310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f36a81c7a60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-templb57317n/artifactsq7syh7s3' '--job-port' '49465' '--artifact-port' '0' '--expansion-port' '0']
E0521 20:28:17.633226026 2305384 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:19 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37001'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46057'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:49465'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_27b83076-6f76-4c47-b71c-7485c0515244.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_27b83076-6f76-4c47-b71c-7485c0515244.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_27b83076-6f76-4c47-b71c-7485c0515244.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_27b83076-6f76-4c47-b71c-7485c0515244.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:20 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0521202820-5617b371_b6379ce4-9daa-4bb5-bc42-b4bf32f4e18e'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:20 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0521202820-5617b371_b6379ce4-9daa-4bb5-bc42-b4bf32f4e18e'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:21 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:21 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0521202820-5617b371_b6379ce4-9daa-4bb5-bc42-b4bf32f4e18e on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38131.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44369.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36707
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521202820-5617b371_b6379ce4-9daa-4bb5-bc42-b4bf32f4e18e: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 20:28:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521202820-5617b371_b6379ce4-9daa-4bb5-bc42-b4bf32f4e18e finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653164904.677310569","description":"Error received from peer ipv6:[::1]:36707","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653164904.677308444","description":"Error received from peer ipv6:[::1]:44369","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653164904.677310569","description":"Error received from peer ipv6:[::1]:36707","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653164904.677308441","description":"Error received from peer ipv6:[::1]:38131","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 59m 54s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/oodlf72nl5daa

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2666

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2666/display/redirect>

Changes:


------------------------------------------
[...truncated 43.19 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0521 14:29:06.592663424 1948970 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653143346
  nanos: 592322349
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653143346
  nanos: 592510700
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653143346
  nanos: 592637300
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653143346
  nanos: 592738389
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653143346
  nanos: 592972993
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653143346
  nanos: 593061685
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 10.384857177734375 seconds with state DONE.
INFO:root:Completed job in 10.384857177734375 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:41359
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb404ac8280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb404ac8310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb404ac8a60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp1_v9uomc/artifactse_b_ia44' '--job-port' '36785' '--artifact-port' '0' '--expansion-port' '0']
E0521 14:29:09.788434252 1949530 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:44289'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33707'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:36785'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_26888aa2-12a2-465e-8ee5-f5354dbc6f83.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_26888aa2-12a2-465e-8ee5-f5354dbc6f83.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_26888aa2-12a2-465e-8ee5-f5354dbc6f83.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_26888aa2-12a2-465e-8ee5-f5354dbc6f83.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:12 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0521142912-5842265_cc3f90d8-8212-409b-80d6-630e7b0d7ae4'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:13 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0521142912-5842265_cc3f90d8-8212-409b-80d6-630e7b0d7ae4'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:13 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:13 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:14 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:14 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:14 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0521142912-5842265_cc3f90d8-8212-409b-80d6-630e7b0d7ae4 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38135.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43437.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46781
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521142912-5842265_cc3f90d8-8212-409b-80d6-630e7b0d7ae4: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 14:29:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521142912-5842265_cc3f90d8-8212-409b-80d6-630e7b0d7ae4 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653143357.401436904","description":"Error received from peer ipv6:[::1]:46781","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653143357.401476109","description":"Error received from peer ipv6:[::1]:38135","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653143357.401435556","description":"Error received from peer ipv6:[::1]:43437","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653143357.401436904","description":"Error received from peer ipv6:[::1]:46781","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 53m 55s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/4jnelzwnwgw7c

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2665

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2665/display/redirect>

Changes:


------------------------------------------
[...truncated 35.85 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0521 08:28:18.297860936 1598129 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653121698
  nanos: 297518491
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653121698
  nanos: 297698974
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653121698
  nanos: 297777175
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653121698
  nanos: 297844409
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653121698
  nanos: 298224210
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653121698
  nanos: 298343896
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 9.710925817489624 seconds with state DONE.
INFO:root:Completed job in 9.710925817489624 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36117
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f26a89c7280> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f26a89c7310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f26a89c7a60> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp54rt5r3u/artifactsrbcqxcy4' '--job-port' '56559' '--artifact-port' '0' '--expansion-port' '0']
E0521 08:28:21.156613489 1598389 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35865'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:36135'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56559'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_edb88c8e-0873-4b5f-baff-926330e22fd7.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_edb88c8e-0873-4b5f-baff-926330e22fd7.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_edb88c8e-0873-4b5f-baff-926330e22fd7.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_edb88c8e-0873-4b5f-baff-926330e22fd7.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:24 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0521082824-5f68393f_3cc049bf-cf15-497f-9275-3ede7448ed77'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:24 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0521082824-5f68393f_3cc049bf-cf15-497f-9275-3ede7448ed77'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:24 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:24 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:25 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:25 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:25 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:25 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0521082824-5f68393f_3cc049bf-cf15-497f-9275-3ede7448ed77 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43987.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45823.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38811
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521082824-5f68393f_3cc049bf-cf15-497f-9275-3ede7448ed77: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 08:28:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521082824-5f68393f_3cc049bf-cf15-497f-9275-3ede7448ed77 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653121708.218136398","description":"Error received from peer ipv6:[::1]:38811","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653121708.218141245","description":"Error received from peer ipv6:[::1]:45823","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653121708.218181834","description":"Error received from peer ipv6:[::1]:43987","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653121708.218136398","description":"Error received from peer ipv6:[::1]:38811","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 58m 18s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/jpl5z7qvdwfcs

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2664

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2664/display/redirect?page=changes>

Changes:

[noreply] [BEAM-13015] Only create a TimerBundleTracker if there are timers.


------------------------------------------
[...truncated 35.95 MB...]
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:772"
thread: "Thread-14"

E0521 02:27:47.617586309 1168103 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:root:severity: INFO
timestamp {
  seconds: 1653100067
  nanos: 617194175
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:261"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653100067
  nanos: 617367982
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:262"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653100067
  nanos: 617451667
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py:805"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653100067
  nanos: 617536544
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:877"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653100067
  nanos: 618577480
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py:274"
thread: "MainThread"

INFO:root:severity: INFO
timestamp {
  seconds: 1653100067
  nanos: 618745565
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py:181"
thread: "MainThread"

INFO:apache_beam.runners.portability.local_job_service:Completed job in 9.878861427307129 seconds with state DONE.
INFO:root:Completed job in 9.878861427307129 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE

> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35649
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.8_sdk:2.40.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f5759e5d310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f5759e5d3a0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f5759e5daf0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.40.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempf7vmpplw/artifactslkduer2h' '--job-port' '49739' '--artifact-port' '0' '--expansion-port' '0']
E0521 02:27:50.669370193 1168379 fork_posix.cc:76]           Other threads are currently calling into gRPC, skipping fork() handlers
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:52 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:52 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35059'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:52 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:45783'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:52 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:49739'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:52 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_424d5088-8c2b-4321-868c-1ab829556823.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_424d5088-8c2b-4321-868c-1ab829556823.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_424d5088-8c2b-4321-868c-1ab829556823.null.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_424d5088-8c2b-4321-868c-1ab829556823.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:54 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0521022754-33d177cc_a42e1e36-e145-43a9-b49d-119c1795a1e7'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:54 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0521022754-33d177cc_a42e1e36-e145-43a9-b49d-119c1795a1e7'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  with Pipeline() as p:
    p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:54 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:54 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:55 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:55 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:55 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0521022754-33d177cc_a42e1e36-e145-43a9-b49d-119c1795a1e7 on Spark master local[4]'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38511.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33623.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37613
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:56 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521022754-33d177cc_a42e1e36-e145-43a9-b49d-119c1795a1e7: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.'
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:b'22/05/21 02:27:57 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0521022754-33d177cc_a42e1e36-e145-43a9-b49d-119c1795a1e7 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 251, in run
    for work_request in self._control_stub.Control(get_responses()):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653100078.262593892","description":"Error received from peer ipv6:[::1]:37613","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1008, in pull_responses
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653100078.262755111","description":"Error received from peer ipv6:[::1]:38511","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    self._target(*self._args, **self._kwargs)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
    for response in responses:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    target=lambda: self._read_inputs(elements_iterator),
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    for elements in elements_iterator:
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 426, in __next__
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1653100078.262715964","description":"Error received from peer ipv6:[::1]:33623","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>
    return self._next()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/grpc/_channel.py",> line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connection reset by peer"
	debug_error_string = "{"created":"@1653100078.262593892","description":"Error received from peer ipv6:[::1]:37613","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Connection reset by peer","grpc_status":14}"
>

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 2h 2m 19s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/qpm6qaqq5a2me

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2663

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2663/display/redirect?page=changes>

Changes:

[yathu] Add labels for typescript PRs

[noreply] Bump google.golang.org/grpc from 1.45.0 to 1.46.2 in /sdks (#17677)


------------------------------------------
[...truncated 44.99 MB...]
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2204: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2519: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2521: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2545: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2194: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/io/gcp/tests/utils.py:100
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/dataframe/io.py:629
apache_beam/dataframe/io.py:629
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:629: FutureWarning: WriteToFiles is experimental.
    return pcoll | fileio.WriteToFiles(

apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2563: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2584: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/bigquery_test.py:1636
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1636: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    streaming = self.test_pipeline.options.view_as(StandardOptions).streaming

apache_beam/io/gcp/bigquery_read_it_test.py:169
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:169: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(

apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
    io.BigQuerySink(

apache_beam/ml/gcp/cloud_dlp_it_test.py:74
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
    | MaskDetectedDetails(

apache_beam/ml/gcp/cloud_dlp_it_test.py:85
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
    | InspectForDetails(

apache_beam/io/gcp/bigquery_read_it_test.py:566
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:566: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery_read_it_test.py:680
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:680: FutureWarning: ReadAllFromBigQuery is experimental.
    | beam.io.ReadAllFromBigQuery())

apache_beam/io/gcp/bigquery.py:2693
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2693: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2694
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2694: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2707
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2707: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    options=pcoll.pipeline.options,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
============ 76 passed, 8 skipped, 207 warnings in 6984.91 seconds =============

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 2h 47s
220 actionable tasks: 157 executed, 57 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/eu64vyznd3slo

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python38 #2662

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python38/2662/display/redirect?page=changes>

Changes:

[bulat.safiullin] [BEAM-14418] added arrows to slider

[noreply] Minor: Bump Dataflow container versions (#17684)


------------------------------------------
[...truncated 45.21 MB...]
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
apache_beam/io/gcp/bigquery.py:2204
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2204: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
apache_beam/io/gcp/bigquery.py:2519
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2519: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
apache_beam/io/gcp/bigquery.py:2521
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2521: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
apache_beam/io/gcp/bigquery.py:2545
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2545: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
apache_beam/io/gcp/bigquery.py:2194
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2194: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
apache_beam/io/gcp/bigquery_file_loads.py:1128
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
apache_beam/io/gcp/bigquery_file_loads.py:1130
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/io/gcp/tests/utils.py:100
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/dataframe/io.py:629
apache_beam/dataframe/io.py:629
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/dataframe/io.py>:629: FutureWarning: WriteToFiles is experimental.
    return pcoll | fileio.WriteToFiles(

apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

apache_beam/io/gcp/bigquery_test.py:1636
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1636: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    streaming = self.test_pipeline.options.view_as(StandardOptions).streaming

apache_beam/io/gcp/bigquery_read_it_test.py:169
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:169: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
apache_beam/io/gcp/bigquery.py:2563
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2563: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
apache_beam/io/gcp/bigquery.py:2584
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2584: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    pipeline_options=pcoll.pipeline.options,

apache_beam/ml/gcp/cloud_dlp_it_test.py:74
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:74: FutureWarning: MaskDetectedDetails is experimental.
    | MaskDetectedDetails(

apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:81
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:81: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(

apache_beam/runners/dataflow/ptransform_overrides.py:316
apache_beam/runners/dataflow/ptransform_overrides.py:316
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:316: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
    io.BigQuerySink(

apache_beam/ml/gcp/cloud_dlp_it_test.py:85
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:85: FutureWarning: InspectForDetails is experimental.
    | InspectForDetails(

apache_beam/io/gcp/bigquery_read_it_test.py:566
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:566: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
    beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))

apache_beam/io/gcp/bigquery_read_it_test.py:680
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:680: FutureWarning: ReadAllFromBigQuery is experimental.
    | beam.io.ReadAllFromBigQuery())

apache_beam/io/gcp/bigquery.py:2693
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2693: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/io/gcp/bigquery.py:2694
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2694: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project

apache_beam/io/gcp/bigquery.py:2707
  <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2707: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    options=pcoll.pipeline.options,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/pytest_postCommitIT-df-py38.xml> -
============ 76 passed, 8 skipped, 207 warnings in 6814.62 seconds =============

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python38/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 314

* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:postCommitPy38IT'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD FAILED in 1h 55m 38s
220 actionable tasks: 150 executed, 64 from cache, 6 up-to-date

Publishing build scan...
https://gradle.com/s/ai4w5fetxf2wg

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org