You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/03/05 01:05:35 UTC
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2592
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2592/display/redirect?page=changes>
Changes:
[boyuanz] Update verify_release_build script to run python tests with dev version.
[jvilcek] [BEAM-9360] Fix equivalence check for FieldType
------------------------------------------
[...truncated 445.62 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 01:05:26 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 01:05:26 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 01:05:26 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ac47a0dd989471ed77633fa9bb84abc7095fedd77a8bddcac817559f63acbdc4. Logs:
2020/03/05 01:05:26 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583370316022226045" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583370316022226045-jenkins-0305010516-62cc38d" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583370316022226045_74f54f3f-30ab-4d8d-907d-c70dd746013a/MANIFEST" logging_endpoint:<url:"localhost:38755" > artifact_endpoint:<url:"localhost:35109" > control_endpoint:<url:"localhost:41785" >
2020/03/05 01:05:26 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:45125
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583370326 nanos:533371228 > message:"forcing DataChannel[localhost:41161] reconnection on port {localhost:41161} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 01:05:27 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-010014"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm ac47a0dd989471ed77633fa9bb84abc7095fedd77a8bddcac817559f63acbdc4'. stderr: Error: No such container: ac47a0dd989471ed77633fa9bb84abc7095fedd77a8bddcac817559f63acbdc4
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 01:05:28 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583370316022226045_74f54f3f-30ab-4d8d-907d-c70dd746013a/MANIFEST
20/03/05 01:05:28 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583370316022226045_74f54f3f-30ab-4d8d-907d-c70dd746013a/MANIFEST -> 1 artifacts
20/03/05 01:05:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 01:05:29 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:44175 ...
20/03/05 01:05:29 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:45189 ...
20/03/05 01:05:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 01:05:29 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:40151 ...
20/03/05 01:05:29 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 01:05:29 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 3970081 ns
20/03/05 01:05:29 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-010014"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 01:05:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 01:05:29 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 01:05:29 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d22dc38a177e48522329161bb3cf5542462d55584fb580a1becb5265f567b826. Logs:
2020/03/05 01:05:28 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583370316022226045" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583370316022226045-jenkins-0305010516-62cc38d" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583370316022226045_74f54f3f-30ab-4d8d-907d-c70dd746013a/MANIFEST" logging_endpoint:<url:"localhost:45189" > artifact_endpoint:<url:"localhost:37849" > control_endpoint:<url:"localhost:44175" >
2020/03/05 01:05:28 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:34045
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583370329 nanos:66057839 > message:"forcing DataChannel[localhost:40151] reconnection on port {localhost:40151} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 01:05:29 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-010014"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm d22dc38a177e48522329161bb3cf5542462d55584fb580a1becb5265f567b826'. stderr: Error: No such container: d22dc38a177e48522329161bb3cf5542462d55584fb580a1becb5265f567b826
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 01:05:29 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583370316022226045-jenkins-0305010516-62cc38d_1f6a6ca7-1ea1-4f3f-8caf-2259f5b6fd44 finished.
20/03/05 01:05:30 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583370316022226045_74f54f3f-30ab-4d8d-907d-c70dd746013a/MANIFEST has 1 artifact locations
20/03/05 01:05:30 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583370316022226045_74f54f3f-30ab-4d8d-907d-c70dd746013a/
2020/03/05 01:05:30 Job state: DONE
2020/03/05 01:05:30 Test flatten:flatten completed
2020/03/05 01:05:30 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-010014" (must force) - container 898208efd435 is using its referenced image bc667ba64ac1
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:9e7d67158b5094e7e12e40b980fe2ec6a2b920d8f7e241008e502e443801bd96
Associated tags:
- 20200305-010014
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-010014
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-010014].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:9e7d67158b5094e7e12e40b980fe2ec6a2b920d8f7e241008e502e443801bd96].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
>>> FAILURE
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 14s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/lyte4yil7en66
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Go_VR_Spark #2608
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2608/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2607
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2607/display/redirect?page=changes>
Changes:
[chuck.yang] Use Avro format for file loads to BigQuery
------------------------------------------
[...truncated 445.58 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 01:11:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 01:11:50 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 01:11:50 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2b6e5cb7585549ae4ee5dbd41ab807bc6ba468af6f6fdb246c7767b361fb351f. Logs:
2020/03/06 01:11:50 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583457097396906625" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583457097396906625-jenkins-0306011138-c9fb6c68" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST" logging_endpoint:<url:"localhost:34359" > artifact_endpoint:<url:"localhost:36151" > control_endpoint:<url:"localhost:46819" >
2020/03/06 01:11:50 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:34503
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583457110 nanos:650448304 > message:"forcing DataChannel[localhost:34141] reconnection on port {localhost:34141} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/06 01:11:51 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 2b6e5cb7585549ae4ee5dbd41ab807bc6ba468af6f6fdb246c7767b361fb351f'. stderr: Error: No such container: 2b6e5cb7585549ae4ee5dbd41ab807bc6ba468af6f6fdb246c7767b361fb351f
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 01:11:52 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST
20/03/06 01:11:52 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST -> 1 artifacts
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:32811 ...
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:42175 ...
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35227 ...
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 5054053 ns
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 01:11:53 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6865b1c5da5f6d598cf545a2c7927c66da7a4a3d488ddfcb75c06a1f200264fa. Logs:
2020/03/06 01:11:52 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583457097396906625" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583457097396906625-jenkins-0306011138-c9fb6c68" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST" logging_endpoint:<url:"localhost:42175" > artifact_endpoint:<url:"localhost:41947" > control_endpoint:<url:"localhost:32811" >
2020/03/06 01:11:52 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:42917
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583457113 nanos:527848417 > message:"forcing DataChannel[localhost:35227] reconnection on port {localhost:35227} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/06 01:11:54 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 6865b1c5da5f6d598cf545a2c7927c66da7a4a3d488ddfcb75c06a1f200264fa'. stderr: Error: No such container: 6865b1c5da5f6d598cf545a2c7927c66da7a4a3d488ddfcb75c06a1f200264fa
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 01:11:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583457097396906625-jenkins-0306011138-c9fb6c68_dd5bfe87-161d-4b82-843a-100255f002c8 finished.
20/03/06 01:11:54 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST has 1 artifact locations
20/03/06 01:11:54 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/
2020/03/06 01:11:54 Job state: DONE
2020/03/06 01:11:54 Test flatten:flatten completed
2020/03/06 01:11:54 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631" (must force) - container 9c27e24ec8cd is using its referenced image f3a86203f6d7
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Failed to remove container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:57db3db7e6817618a03864efd77ad8c34385cc326246e77ee77c6fccf631b28c
Associated tags:
- 20200306-010631
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:57db3db7e6817618a03864efd77ad8c34385cc326246e77ee77c6fccf631b28c].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
>>> FAILURE
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 30s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/jk63ot3yswqcy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2606
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2606/display/redirect>
Changes:
------------------------------------------
[...truncated 447.62 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 00:09:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 00:09:15 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 00:09:15 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2d0b83d6cc58e81f490748b911358558d7a3218fedcd434610fb390cb7215f43. Logs:
2020/03/06 00:09:15 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583453337143350768" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583453337143350768-jenkins-0306000857-44df8858" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST" logging_endpoint:<url:"localhost:40443" > artifact_endpoint:<url:"localhost:36355" > control_endpoint:<url:"localhost:32931" >
2020/03/06 00:09:15 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:44975
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583453355 nanos:606111561 > message:"forcing DataChannel[localhost:37089] reconnection on port {localhost:37089} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/06 00:09:16 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 2d0b83d6cc58e81f490748b911358558d7a3218fedcd434610fb390cb7215f43'. stderr: Error: No such container: 2d0b83d6cc58e81f490748b911358558d7a3218fedcd434610fb390cb7215f43
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST -> 1 artifacts
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:39315 ...
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41613 ...
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35527 ...
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 3193227 ns
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 00:09:20 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c9a67ed6a767359909d62bfd7989f5c71d01d54966a5a02b878f4e5c12621f4a. Logs:
2020/03/06 00:09:20 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583453337143350768" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583453337143350768-jenkins-0306000857-44df8858" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST" logging_endpoint:<url:"localhost:41613" > artifact_endpoint:<url:"localhost:37715" > control_endpoint:<url:"localhost:39315" >
2020/03/06 00:09:20 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:42257
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583453360 nanos:342237218 > message:"forcing DataChannel[localhost:35527] reconnection on port {localhost:35527} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/06 00:09:21 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm c9a67ed6a767359909d62bfd7989f5c71d01d54966a5a02b878f4e5c12621f4a'. stderr: Error: No such container: c9a67ed6a767359909d62bfd7989f5c71d01d54966a5a02b878f4e5c12621f4a
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 00:09:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583453337143350768-jenkins-0306000857-44df8858_0618ec9b-3083-48e9-b12b-6c3de7a5dc32 finished.
20/03/06 00:09:21 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST has 1 artifact locations
20/03/06 00:09:21 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/
2020/03/06 00:09:21 Job state: DONE
2020/03/06 00:09:21 Test flatten:flatten completed
2020/03/06 00:09:21 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233" (must force) - container 939e11895d10 is using its referenced image 5ad656894196
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:5b10dc23af8408ed8bca9af7d40f7f403ff72a9e3dbd65038754d1a7957b04cc
Associated tags:
- 20200306-000233
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:5b10dc23af8408ed8bca9af7d40f7f403ff72a9e3dbd65038754d1a7957b04cc].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 1s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/iwz7xm3wrn23g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2605
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2605/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-9450] Update www.apache.org/dist/ links to downloads.apache.org
[iemejia] [BEAM-9450] Convert links available via https to use https
------------------------------------------
[...truncated 164.74 KB...]
>
pcollections: <
key: "n3"
value: <
unique_name: "n3"
coder_id: "c0"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n4"
value: <
unique_name: "n4"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n5"
value: <
unique_name: "n5"
coder_id: "c0"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n6"
value: <
unique_name: "n6"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n7"
value: <
unique_name: "n7"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n8"
value: <
unique_name: "n8"
coder_id: "c4"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n9"
value: <
unique_name: "n9"
coder_id: "c6"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
windowing_strategies: <
key: "w0"
value: <
window_fn: <
urn: "beam:window_fn:global_windows:v1"
>
merge_status: NON_MERGING
window_coder_id: "c1"
trigger: <
default: <
>
>
accumulation_mode: DISCARDING
output_time: END_OF_WINDOW
closing_behavior: EMIT_IF_NONEMPTY
OnTimeBehavior: FIRE_ALWAYS
environment_id: "go"
>
>
coders: <
key: "c0"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c1"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:go:coder:custom:v1"
payload: "Cgd2YXJpbnR6EgIIAhqFAQpxZ2l0aHViLmNvbS9hcGFjaGUvYmVhbS9zZGtzL2dvL3Rlc3QvdmVuZG9yL2dpdGh1Yi5jb20vYXBhY2hlL2JlYW0vc2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvY29kZXJ4LmVuY1ZhckludFoSEAgWIgQIGUAPKgYIFBICCAgikQEKcWdpdGh1Yi5jb20vYXBhY2hlL2JlYW0vc2Rrcy9nby90ZXN0L3ZlbmRvci9naXRodWIuY29tL2FwYWNoZS9iZWFtL3Nka3MvZ28vcGtnL2JlYW0vY29yZS9ydW50aW1lL2NvZGVyeC5kZWNWYXJJbnRaEhwIFiIECBlAAyIGCBQSAggIKgQIGUAPKgQIGUAB"
>
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c3"
component_coder_ids: "c3"
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c3"
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c3"
component_coder_ids: "c5"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
>
>
>
root_transform_ids: "e5"
root_transform_ids: "e3"
root_transform_ids: "e4"
root_transform_ids: "e1"
root_transform_ids: "e2"
root_transform_ids: "e6"
root_transform_ids: "e7"
root_transform_ids: "s1"
2020/03/05 22:02:14 Test flatten:flatten failed: connecting to job service:
failed to dial server at localhost:60151
caused by:
context deadline exceeded
2020/03/05 22:02:14 Result: 7 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:67223f1419626f15845dfcdaa0357ecf30ef483fcb31d9852c6b7cdee15e1d68
Deleted: sha256:040b4978b5b62938938d75551d31ef6b4f506cc96efb7b253aca74c560568f09
Deleted: sha256:f2ccdb331bad0513f6794488dcf1cdd9e7b0ad18815f2f326584235545e836c5
Deleted: sha256:71af76733efbc7a3d399c36900a4f21f4824beee3c414adcea4fe1534e56960f
Deleted: sha256:baa68cea33770e5564ec9a72aaad9aed3053a0d9c58da57a49e8dff5a62f3e7f
Deleted: sha256:0d3f9e843e0718adbd2f52e1cd48ca4a100a7f3e50e6a56b26c799c40ffb36d4
Deleted: sha256:b16063a781f21e2a03d01bffb7c8315d8870fe9f71e06ed5e7c340bda2db8a99
Deleted: sha256:0e8ed0424902cbb715b9f0cb54c3ff72da9965680b2fcdb961780b5e274282d8
Deleted: sha256:ddd28c44250c585cf7df4c03c2736efdd021d7d2dd156bf5fbeeda98532dcbe9
Deleted: sha256:8e41a512105114d0e5d1ca1b8d80087ef1c8a41e04ef356d85167af2ae53685f
Deleted: sha256:d8f086eeeb6aacd554b2261f7cf9e57e71bc0b6209690aee8d6ea37a93456fae
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:67223f1419626f15845dfcdaa0357ecf30ef483fcb31d9852c6b7cdee15e1d68
Associated tags:
- 20200305-214711
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:67223f1419626f15845dfcdaa0357ecf30ef483fcb31d9852c6b7cdee15e1d68].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 18m 27s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/rg25dyyy4mm22
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2604
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2604/display/redirect?page=changes>
Changes:
[git] Remove optionality and add sensible defaults to PubsubIO builders.
------------------------------------------
[...truncated 445.84 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 21:09:25 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 21:09:25 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 21:09:25 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 585e325420f01e1ce799b163d22d8439f979f7f2d6712de4e9f3af2b0671fe5d. Logs:
2020/03/05 21:09:25 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583442557218548443" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583442557218548443-jenkins-0305210917-8b42e2d" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST" logging_endpoint:<url:"localhost:35671" > artifact_endpoint:<url:"localhost:32937" > control_endpoint:<url:"localhost:46171" >
2020/03/05 21:09:25 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:39699
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583442565 nanos:842808066 > message:"forcing DataChannel[localhost:46477] reconnection on port {localhost:46477} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 21:09:26 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 585e325420f01e1ce799b163d22d8439f979f7f2d6712de4e9f3af2b0671fe5d'. stderr: Error: No such container: 585e325420f01e1ce799b163d22d8439f979f7f2d6712de4e9f3af2b0671fe5d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 21:09:27 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST
20/03/05 21:09:27 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST -> 1 artifacts
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35009 ...
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:34081 ...
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41901 ...
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2800435 ns
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 21:09:28 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 903b8f5367c2cdc1034e6247bc2ed159a2222d45824c393352d1c8c990679a87. Logs:
2020/03/05 21:09:27 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583442557218548443" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583442557218548443-jenkins-0305210917-8b42e2d" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST" logging_endpoint:<url:"localhost:34081" > artifact_endpoint:<url:"localhost:42371" > control_endpoint:<url:"localhost:35009" >
2020/03/05 21:09:27 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:33497
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583442568 nanos:116743726 > message:"forcing DataChannel[localhost:41901] reconnection on port {localhost:41901} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 21:09:28 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 903b8f5367c2cdc1034e6247bc2ed159a2222d45824c393352d1c8c990679a87'. stderr: Error: No such container: 903b8f5367c2cdc1034e6247bc2ed159a2222d45824c393352d1c8c990679a87
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 21:09:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583442557218548443-jenkins-0305210917-8b42e2d_722ca76d-5040-4a4a-9188-0d0c67db4f7b finished.
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST has 1 artifact locations
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/
2020/03/05 21:09:28 Job state: DONE
2020/03/05 21:09:28 Test flatten:flatten completed
2020/03/05 21:09:28 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335" (must force) - container dceef77112d2 is using its referenced image 8cdd34626838
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:c192848e6b16c5452c2388cc2e4b1fd73fcc297ee4f54581c4e285f1ecb9c1e7
Associated tags:
- 20200305-210335
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:c192848e6b16c5452c2388cc2e4b1fd73fcc297ee4f54581c4e285f1ecb9c1e7].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
>>> FAILURE
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 57s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/i2kx5blg7wkai
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2603
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2603/display/redirect?page=changes>
Changes:
[jkai] [Hotfix] fix rabbitmp spotless check
------------------------------------------
[...truncated 447.45 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 20:19:48 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 20:19:48 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 20:19:48 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container af62669adcb526746847b11625ad132e2048e13b517882573f370b232f65ec41. Logs:
2020/03/05 20:19:47 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583439579607091859" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583439579607091859-jenkins-0305201940-e221b732" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST" logging_endpoint:<url:"localhost:33023" > artifact_endpoint:<url:"localhost:40233" > control_endpoint:<url:"localhost:43991" >
2020/03/05 20:19:47 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:33025
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583439588 nanos:235425853 > message:"forcing DataChannel[localhost:39719] reconnection on port {localhost:39719} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 20:19:48 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm af62669adcb526746847b11625ad132e2048e13b517882573f370b232f65ec41'. stderr: Error: No such container: af62669adcb526746847b11625ad132e2048e13b517882573f370b232f65ec41
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST -> 1 artifacts
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35697 ...
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33571 ...
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41689 ...
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2826109 ns
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 20:19:50 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 98a4c3bf32e85cf35e9d536ca575f9211002c5935d99cd3d7c5be46fedc055b3. Logs:
2020/03/05 20:19:50 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583439579607091859" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583439579607091859-jenkins-0305201940-e221b732" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST" logging_endpoint:<url:"localhost:33571" > artifact_endpoint:<url:"localhost:40965" > control_endpoint:<url:"localhost:35697" >
2020/03/05 20:19:50 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:39935
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583439590 nanos:321899958 > message:"forcing DataChannel[localhost:41689] reconnection on port {localhost:41689} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 20:19:51 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 98a4c3bf32e85cf35e9d536ca575f9211002c5935d99cd3d7c5be46fedc055b3'. stderr: Error: No such container: 98a4c3bf32e85cf35e9d536ca575f9211002c5935d99cd3d7c5be46fedc055b3
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 20:19:51 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583439579607091859-jenkins-0305201940-e221b732_c6a6be5d-26fc-48e4-9ccc-540606bfb7da finished.
20/03/05 20:19:51 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST has 1 artifact locations
20/03/05 20:19:51 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/
2020/03/05 20:19:51 Job state: DONE
2020/03/05 20:19:51 Test flatten:flatten completed
2020/03/05 20:19:51 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434" (must force) - container 0e063d24ebad is using its referenced image 8ab8d3c9143d
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:426fd7ae8af07c913cfdaea8d1edbecc6a76e2fc9b184a6386c9d4289d8ace57
Associated tags:
- 20200305-201434
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:426fd7ae8af07c913cfdaea8d1edbecc6a76e2fc9b184a6386c9d4289d8ace57].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
>>> FAILURE
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 20s
55 actionable tasks: 42 executed, 13 from cache
Publishing build scan...
https://gradle.com/s/ookdqbxna7fns
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2602
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2602/display/redirect?page=changes>
Changes:
[amaliujia] [BEAM-9288] Not bundle conscrypt in gRPC vendor in META-INF/
[lcwik] [BEAM-9452] Update classgraph to latest version to resolve windows
------------------------------------------
[...truncated 458.61 KB...]
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:125)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:159)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:201)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:185)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 32 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error response from daemon: Cannot kill container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d: No such container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:148)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:186)
... 42 more
20/03/05 18:57:22 ERROR org.apache.spark.scheduler.TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
20/03/05 18:57:22 ERROR org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation: Error during job invocation go0job0701583434627048740453-jenkins-0305185707-de3c0e7c_ddcb107c-8f98-4ae0-ad9f-747169145594.
org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:71)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:102)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:90)
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:139)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:333)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:321)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:251)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.getStageBundleFactory(DefaultExecutableStageContext.java:38)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingExecutableStageContextFactory.java:198)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:125)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
... 3 more
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:159)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:201)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:185)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 32 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error response from daemon: Cannot kill container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d: No such container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:148)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:186)
... 42 more
2020/03/05 18:57:22 (): org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:71)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:102)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:90)
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:139)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:333)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:321)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:251)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.getStageBundleFactory(DefaultExecutableStageContext.java:38)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingExecutableStageContextFactory.java:198)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:125)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
... 3 more
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:159)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:201)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:185)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 32 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error response from daemon: Cannot kill container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d: No such container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:148)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:186)
... 42 more
2020/03/05 18:57:22 (): java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
20/03/05 18:57:22 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583434627048740453_4cf6a1ce-0f09-4780-b451-04932fc37444/MANIFEST has 1 artifact locations
20/03/05 18:57:22 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583434627048740453_4cf6a1ce-0f09-4780-b451-04932fc37444/
2020/03/05 18:57:22 Job state: FAILED
2020/03/05 18:57:22 Test flatten:flatten failed: job go0job0701583434627048740453-jenkins-0305185707-de3c0e7c_ddcb107c-8f98-4ae0-ad9f-747169145594 failed
2020/03/05 18:57:22 Result: 2 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-185143" (must force) - container b45cd7f62242 is using its referenced image 965c9ef327b8
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:9714b326adea54e228bd5d9aac920f7526a0fca99fc5c43a0428db0f0dce6e7c
Associated tags:
- 20200305-185143
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-185143
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-185143].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:9714b326adea54e228bd5d9aac920f7526a0fca99fc5c43a0428db0f0dce6e7c].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
>>> FAILURE
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 27s
55 actionable tasks: 54 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/xlbpqf6ivs2ra
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2601
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2601/display/redirect>
Changes:
------------------------------------------
[...truncated 447.21 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 18:11:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 18:11:14 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 18:11:14 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e45e0c28ddc26d142d28a2dc4cdda0ddb4bd9e979dea2556df0a3e8bd6e26d14. Logs:
2020/03/05 18:11:14 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583431865548803456" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583431865548803456-jenkins-0305181105-40e21032" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST" logging_endpoint:<url:"localhost:45727" > artifact_endpoint:<url:"localhost:33779" > control_endpoint:<url:"localhost:35607" >
2020/03/05 18:11:14 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:39599
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583431874 nanos:464957596 > message:"forcing DataChannel[localhost:41825] reconnection on port {localhost:41825} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 18:11:15 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm e45e0c28ddc26d142d28a2dc4cdda0ddb4bd9e979dea2556df0a3e8bd6e26d14'. stderr: Error: No such container: e45e0c28ddc26d142d28a2dc4cdda0ddb4bd9e979dea2556df0a3e8bd6e26d14
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST -> 1 artifacts
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:34145 ...
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33233 ...
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:42803 ...
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2708295 ns
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 18:11:16 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 889fc62c7dec204013ca12a5c62eae97a604908aa5c67e92af1f8e38f804644a. Logs:
2020/03/05 18:11:16 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583431865548803456" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583431865548803456-jenkins-0305181105-40e21032" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST" logging_endpoint:<url:"localhost:33233" > artifact_endpoint:<url:"localhost:36191" > control_endpoint:<url:"localhost:34145" >
2020/03/05 18:11:16 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:33395
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583431876 nanos:752070044 > message:"forcing DataChannel[localhost:42803] reconnection on port {localhost:42803} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 18:11:17 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 889fc62c7dec204013ca12a5c62eae97a604908aa5c67e92af1f8e38f804644a'. stderr: Error: No such container: 889fc62c7dec204013ca12a5c62eae97a604908aa5c67e92af1f8e38f804644a
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 18:11:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583431865548803456-jenkins-0305181105-40e21032_5cdd34f9-c288-4838-af15-f67ebca1ad44 finished.
20/03/05 18:11:17 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST has 1 artifact locations
20/03/05 18:11:17 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/
2020/03/05 18:11:17 Job state: DONE
2020/03/05 18:11:17 Test flatten:flatten completed
2020/03/05 18:11:17 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601" (must force) - container f5f1fb296fb6 is using its referenced image 4f396679cc4e
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:3915d59fa31148c53ed69a293aa9e8ee88ab55bfba6850a3a674a60ba331e87a
Associated tags:
- 20200305-180601
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:3915d59fa31148c53ed69a293aa9e8ee88ab55bfba6850a3a674a60ba331e87a].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 5s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/qynppn4tgpfj6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2600
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2600/display/redirect?page=changes>
Changes:
[hktang] [BEAM-9453] Changed new string creation to use StandardCharsets.UTF_8
------------------------------------------
[...truncated 446.50 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 17:45:35 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 17:45:35 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 17:45:35 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1c439a64d455133af9ef8ce2e31c1d3a3cf9769deb99e25862c7828c3773bd5c. Logs:
2020/03/05 17:45:35 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583430326975248778" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583430326975248778-jenkins-0305174527-7fd4806f" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST" logging_endpoint:<url:"localhost:40437" > artifact_endpoint:<url:"localhost:35343" > control_endpoint:<url:"localhost:42231" >
2020/03/05 17:45:35 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:42579
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583430335 nanos:700689327 > message:"forcing DataChannel[localhost:44581] reconnection on port {localhost:44581} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 17:45:36 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 1c439a64d455133af9ef8ce2e31c1d3a3cf9769deb99e25862c7828c3773bd5c'. stderr: Error: No such container: 1c439a64d455133af9ef8ce2e31c1d3a3cf9769deb99e25862c7828c3773bd5c
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST -> 1 artifacts
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:34479 ...
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:32807 ...
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33041 ...
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2628866 ns
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 17:45:37 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f17032c2be6020422e39011c834faa6caeef2f207ec3489b1c9be730c3a53fd5. Logs:
2020/03/05 17:45:37 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583430326975248778" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583430326975248778-jenkins-0305174527-7fd4806f" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST" logging_endpoint:<url:"localhost:32807" > artifact_endpoint:<url:"localhost:39973" > control_endpoint:<url:"localhost:34479" >
2020/03/05 17:45:37 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:43559
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583430337 nanos:933836329 > message:"forcing DataChannel[localhost:33041] reconnection on port {localhost:33041} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 17:45:38 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm f17032c2be6020422e39011c834faa6caeef2f207ec3489b1c9be730c3a53fd5'. stderr: Error: No such container: f17032c2be6020422e39011c834faa6caeef2f207ec3489b1c9be730c3a53fd5
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 17:45:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583430326975248778-jenkins-0305174527-7fd4806f_1a79ba9a-1a6e-447d-86ed-ed062ab7cbb3 finished.
20/03/05 17:45:38 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST has 1 artifact locations
20/03/05 17:45:38 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/
2020/03/05 17:45:38 Job state: DONE
2020/03/05 17:45:38 Test flatten:flatten completed
2020/03/05 17:45:38 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001" (must force) - container 570faffde618 is using its referenced image fb94576c7e68
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:38ea694cadd831c30d3ebac35986e69a074ecc3bbffa60f7c1141f61594a1952
Associated tags:
- 20200305-174001
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:38ea694cadd831c30d3ebac35986e69a074ecc3bbffa60f7c1141f61594a1952].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 51s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/mxepcgok5ip66
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2599
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2599/display/redirect>
Changes:
------------------------------------------
[...truncated 445.67 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 12:07:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 12:07:42 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 12:07:42 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9af00c6b81c7db2daa8c1784b67f7641d730718fd41e0e9f95c602d7dd154a07. Logs:
2020/03/05 12:07:42 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583410048110577803" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583410048110577803-jenkins-0305120728-71b9f994" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST" logging_endpoint:<url:"localhost:38005" > artifact_endpoint:<url:"localhost:34527" > control_endpoint:<url:"localhost:39805" >
2020/03/05 12:07:42 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:37661
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583410062 nanos:622736736 > message:"forcing DataChannel[localhost:41721] reconnection on port {localhost:41721} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 12:07:43 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 9af00c6b81c7db2daa8c1784b67f7641d730718fd41e0e9f95c602d7dd154a07'. stderr: Error: No such container: 9af00c6b81c7db2daa8c1784b67f7641d730718fd41e0e9f95c602d7dd154a07
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST -> 1 artifacts
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35383 ...
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:38253 ...
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:37151 ...
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 3788293 ns
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 12:07:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 12:07:45 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1cc41f99352256425c0860364c14ff4438416b176f68b196b8a49b0627dcb831. Logs:
2020/03/05 12:07:44 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583410048110577803" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583410048110577803-jenkins-0305120728-71b9f994" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST" logging_endpoint:<url:"localhost:38253" > artifact_endpoint:<url:"localhost:42449" > control_endpoint:<url:"localhost:35383" >
2020/03/05 12:07:44 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:46181
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583410064 nanos:981937259 > message:"forcing DataChannel[localhost:37151] reconnection on port {localhost:37151} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 12:07:45 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 1cc41f99352256425c0860364c14ff4438416b176f68b196b8a49b0627dcb831'. stderr: Error: No such container: 1cc41f99352256425c0860364c14ff4438416b176f68b196b8a49b0627dcb831
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 12:07:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583410048110577803-jenkins-0305120728-71b9f994_eb231148-1c04-4a0c-aac5-8a6bbd8eb074 finished.
20/03/05 12:07:45 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST has 1 artifact locations
20/03/05 12:07:45 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/
2020/03/05 12:07:45 Job state: DONE
2020/03/05 12:07:45 Test flatten:flatten completed
2020/03/05 12:07:45 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222" (must force) - container ea8f75f6c0d0 is using its referenced image 8d7199a204c5
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:59dffdefc146173c01312f6638b520404d911df8e970d22bbbb075b117933b78
Associated tags:
- 20200305-120222
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:59dffdefc146173c01312f6638b520404d911df8e970d22bbbb075b117933b78].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 29s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/5qj4tgx5hynlw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2598
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2598/display/redirect?page=changes>
Changes:
[github] Add integration test for AnnotateVideoWithContext transform (#10986)
------------------------------------------
[...truncated 446.56 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 11:33:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 11:33:37 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 11:33:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c48277b856e7ed286548f439a8cfc78a9f1cd4492d75ed4c1f6374f34788d8d4. Logs:
2020/03/05 11:33:37 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583407984321584619" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583407984321584619-jenkins-0305113304-e7d372b3" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST" logging_endpoint:<url:"localhost:41705" > artifact_endpoint:<url:"localhost:39849" > control_endpoint:<url:"localhost:43203" >
2020/03/05 11:33:37 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:34507
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583408017 nanos:376491270 > message:"forcing DataChannel[localhost:35103] reconnection on port {localhost:35103} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 11:33:38 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm c48277b856e7ed286548f439a8cfc78a9f1cd4492d75ed4c1f6374f34788d8d4'. stderr: Error: No such container: c48277b856e7ed286548f439a8cfc78a9f1cd4492d75ed4c1f6374f34788d8d4
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST -> 1 artifacts
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41237 ...
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33965 ...
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41311 ...
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2822291 ns
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 11:33:39 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container caaf1004eed076999efd1efe95788da2d5fab2b33500fee7b1cbe2b353c50abe. Logs:
2020/03/05 11:33:39 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583407984321584619" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583407984321584619-jenkins-0305113304-e7d372b3" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST" logging_endpoint:<url:"localhost:33965" > artifact_endpoint:<url:"localhost:45949" > control_endpoint:<url:"localhost:41237" >
2020/03/05 11:33:39 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:38397
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583408019 nanos:658509719 > message:"forcing DataChannel[localhost:41311] reconnection on port {localhost:41311} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 11:33:40 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm caaf1004eed076999efd1efe95788da2d5fab2b33500fee7b1cbe2b353c50abe'. stderr: Error: No such container: caaf1004eed076999efd1efe95788da2d5fab2b33500fee7b1cbe2b353c50abe
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 11:33:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583407984321584619-jenkins-0305113304-e7d372b3_c75f545a-e720-40d6-84ec-8f4d5ba2e8f3 finished.
20/03/05 11:33:40 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST has 1 artifact locations
20/03/05 11:33:40 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/
2020/03/05 11:33:40 Job state: DONE
2020/03/05 11:33:40 Test flatten:flatten completed
2020/03/05 11:33:40 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704" (must force) - container 6840f938b5cf is using its referenced image 61da195ceb6b
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:ea80e2a5495966600332a5041b1ea4d78b35ca5062aa10c5193e41ddfd542125
Associated tags:
- 20200305-112704
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:ea80e2a5495966600332a5041b1ea4d78b35ca5062aa10c5193e41ddfd542125].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
>>> FAILURE
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 40s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/b76i3flszgtyw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2597
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2597/display/redirect?page=changes>
Changes:
[github] Update lostluck's info on the Go SDK roadmap
[github] Switch contact email to apache.org.
------------------------------------------
[...truncated 445.78 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 10:05:34 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 10:05:34 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 10:05:34 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 21a0acef3dfbe85dd6746de10bb97d26b09f2957f5f1b8056eaabf9cb4ba0228. Logs:
2020/03/05 10:05:33 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583402725592268491" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583402725592268491-jenkins-0305100526-d51d7bf6" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583402725592268491_f5efbdfe-fb31-475d-9c9e-af5eacec3d5f/MANIFEST" logging_endpoint:<url:"localhost:37271" > artifact_endpoint:<url:"localhost:37235" > control_endpoint:<url:"localhost:43765" >
2020/03/05 10:05:33 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:33093
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583402734 nanos:205394144 > message:"forcing DataChannel[localhost:43357] reconnection on port {localhost:43357} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 10:05:34 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-100016"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 21a0acef3dfbe85dd6746de10bb97d26b09f2957f5f1b8056eaabf9cb4ba0228'. stderr: Error: No such container: 21a0acef3dfbe85dd6746de10bb97d26b09f2957f5f1b8056eaabf9cb4ba0228
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583402725592268491_f5efbdfe-fb31-475d-9c9e-af5eacec3d5f/MANIFEST
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583402725592268491_f5efbdfe-fb31-475d-9c9e-af5eacec3d5f/MANIFEST -> 1 artifacts
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 10:05:36 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:45893 ...
20/03/05 10:05:36 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:42961 ...
20/03/05 10:05:36 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:44527 ...
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 10:05:36 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 4279803 ns
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-100016"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 10:05:36 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 10:05:36 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 64db1de3b60caec57056919b85f47e9992ad9b59e05d48be7c49f44dd9dd01ea. Logs:
2020/03/05 10:05:36 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583402725592268491" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583402725592268491-jenkins-0305100526-d51d7bf6" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583402725592268491_f5efbdfe-fb31-475d-9c9e-af5eacec3d5f/MANIFEST" logging_endpoint:<url:"localhost:42961" > artifact_endpoint:<url:"localhost:40781" > control_endpoint:<url:"localhost:45893" >
2020/03/05 10:05:36 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:43303
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583402736 nanos:448214832 > message:"forcing DataChannel[localhost:44527] reconnection on port {localhost:44527} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 10:05:37 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-100016"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 64db1de3b60caec57056919b85f47e9992ad9b59e05d48be7c49f44dd9dd01ea'. stderr: Error: No such container: 64db1de3b60caec57056919b85f47e9992ad9b59e05d48be7c49f44dd9dd01ea
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 10:05:37 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583402725592268491-jenkins-0305100526-d51d7bf6_0ac36528-f194-49de-a64c-b0a6b844cdae finished.
20/03/05 10:05:37 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583402725592268491_f5efbdfe-fb31-475d-9c9e-af5eacec3d5f/MANIFEST has 1 artifact locations
20/03/05 10:05:37 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583402725592268491_f5efbdfe-fb31-475d-9c9e-af5eacec3d5f/
2020/03/05 10:05:37 Job state: DONE
2020/03/05 10:05:37 Test flatten:flatten completed
2020/03/05 10:05:37 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-100016" (must force) - container 73ed1c761a1f is using its referenced image 132dddaf110a
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:2036c68ccd479f8c582f923d5adb5034a047c342c9d3ff8b34764da4dd7a964b
Associated tags:
- 20200305-100016
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-100016
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-100016].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:2036c68ccd479f8c582f923d5adb5034a047c342c9d3ff8b34764da4dd7a964b].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
>>> FAILURE
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 26s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/ujjnsmsrmkiug
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2596
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2596/display/redirect?page=changes>
Changes:
[github] [BEAM-8328] Disable community metrics integration test in 'test' task
------------------------------------------
[...truncated 446.43 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 09:04:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 09:04:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 09:04:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 80536b7d1870fe0e81f96bdd7496727072672bc2a0c8c43df9fb83245b86355a. Logs:
2020/03/05 09:04:44 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583399076552690524" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583399076552690524-jenkins-0305090436-b5f3cfd4" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583399076552690524_d3b4937c-7d9f-495a-bdd5-ffedde499a01/MANIFEST" logging_endpoint:<url:"localhost:35031" > artifact_endpoint:<url:"localhost:35599" > control_endpoint:<url:"localhost:38569" >
2020/03/05 09:04:44 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:36603
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583399084 nanos:830951649 > message:"forcing DataChannel[localhost:36589] reconnection on port {localhost:36589} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 09:04:45 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-085935"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 80536b7d1870fe0e81f96bdd7496727072672bc2a0c8c43df9fb83245b86355a'. stderr: Error: No such container: 80536b7d1870fe0e81f96bdd7496727072672bc2a0c8c43df9fb83245b86355a
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583399076552690524_d3b4937c-7d9f-495a-bdd5-ffedde499a01/MANIFEST
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583399076552690524_d3b4937c-7d9f-495a-bdd5-ffedde499a01/MANIFEST -> 1 artifacts
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 09:04:46 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:36927 ...
20/03/05 09:04:46 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:40611 ...
20/03/05 09:04:46 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:42095 ...
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 09:04:46 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2548216 ns
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-085935"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 09:04:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 09:04:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e051a268733da33fff21ed824558f371638e9e635cf62e04ae503dac2bb4fac0. Logs:
2020/03/05 09:04:46 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583399076552690524" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583399076552690524-jenkins-0305090436-b5f3cfd4" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583399076552690524_d3b4937c-7d9f-495a-bdd5-ffedde499a01/MANIFEST" logging_endpoint:<url:"localhost:40611" > artifact_endpoint:<url:"localhost:36057" > control_endpoint:<url:"localhost:36927" >
2020/03/05 09:04:46 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:41979
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583399086 nanos:892425949 > message:"forcing DataChannel[localhost:42095] reconnection on port {localhost:42095} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 09:04:47 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-085935"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm e051a268733da33fff21ed824558f371638e9e635cf62e04ae503dac2bb4fac0'. stderr: Error: No such container: e051a268733da33fff21ed824558f371638e9e635cf62e04ae503dac2bb4fac0
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 09:04:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583399076552690524-jenkins-0305090436-b5f3cfd4_0db88e41-cd0e-4d70-be70-59b1fc7d2384 finished.
20/03/05 09:04:47 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583399076552690524_d3b4937c-7d9f-495a-bdd5-ffedde499a01/MANIFEST has 1 artifact locations
20/03/05 09:04:47 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583399076552690524_d3b4937c-7d9f-495a-bdd5-ffedde499a01/
2020/03/05 09:04:47 Job state: DONE
2020/03/05 09:04:47 Test flatten:flatten completed
2020/03/05 09:04:47 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-085935" (must force) - container b7e8027f65ce is using its referenced image 21a4f00a6a58
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:1f56c82a43e60e7d6e9ff52d6fb915c3386e40733bf95ca265ce5bfea3e154cb
Associated tags:
- 20200305-085935
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-085935
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-085935].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:1f56c82a43e60e7d6e9ff52d6fb915c3386e40733bf95ca265ce5bfea3e154cb].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 13s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/cloknqc23q43u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2595
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2595/display/redirect>
Changes:
------------------------------------------
[...truncated 165.28 KB...]
>
pcollections: <
key: "n3"
value: <
unique_name: "n3"
coder_id: "c0"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n4"
value: <
unique_name: "n4"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n5"
value: <
unique_name: "n5"
coder_id: "c0"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n6"
value: <
unique_name: "n6"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n7"
value: <
unique_name: "n7"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n8"
value: <
unique_name: "n8"
coder_id: "c4"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n9"
value: <
unique_name: "n9"
coder_id: "c6"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
windowing_strategies: <
key: "w0"
value: <
window_fn: <
urn: "beam:window_fn:global_windows:v1"
>
merge_status: NON_MERGING
window_coder_id: "c1"
trigger: <
default: <
>
>
accumulation_mode: DISCARDING
output_time: END_OF_WINDOW
closing_behavior: EMIT_IF_NONEMPTY
OnTimeBehavior: FIRE_ALWAYS
environment_id: "go"
>
>
coders: <
key: "c0"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c1"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:go:coder:custom:v1"
payload: "Cgd2YXJpbnR6EgIIAhqFAQpxZ2l0aHViLmNvbS9hcGFjaGUvYmVhbS9zZGtzL2dvL3Rlc3QvdmVuZG9yL2dpdGh1Yi5jb20vYXBhY2hlL2JlYW0vc2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvY29kZXJ4LmVuY1ZhckludFoSEAgWIgQIGUAPKgYIFBICCAgikQEKcWdpdGh1Yi5jb20vYXBhY2hlL2JlYW0vc2Rrcy9nby90ZXN0L3ZlbmRvci9naXRodWIuY29tL2FwYWNoZS9iZWFtL3Nka3MvZ28vcGtnL2JlYW0vY29yZS9ydW50aW1lL2NvZGVyeC5kZWNWYXJJbnRaEhwIFiIECBlAAyIGCBQSAggIKgQIGUAPKgQIGUAB"
>
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c3"
component_coder_ids: "c3"
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c3"
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c3"
component_coder_ids: "c5"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-060328"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
>
>
>
root_transform_ids: "e5"
root_transform_ids: "e1"
root_transform_ids: "e2"
root_transform_ids: "e6"
root_transform_ids: "e3"
root_transform_ids: "e4"
root_transform_ids: "e7"
root_transform_ids: "s1"
2020/03/05 06:18:34 Test flatten:flatten failed: connecting to job service:
failed to dial server at localhost:43853
caused by:
context deadline exceeded
2020/03/05 06:18:34 Result: 7 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-060328
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:5b2ae7f2f6236e19e76398e7d6b65d13a5540b9333b4573697f77fc495e4b4a9
Deleted: sha256:0430ea4f94eab13cbeaf5c770f5356fc300e102f110e1128d52e24912c3f1302
Deleted: sha256:61382f830109179406cbd16d6b551145875230619a1f549ee6204b70bbf216a4
Deleted: sha256:8083630ddd2700eb068abdf00a6d3963d19ed97e7ab4a741e2e76408e16aa218
Deleted: sha256:438eb3f7d1511040ba5842e1d3f0f3615d1abe4e449a033966b4cd6d17c5ec58
Deleted: sha256:db415ea34d0993cfaed6bec655adaa03460cf25c4742142c1d44a5c2cf7526c5
Deleted: sha256:5c770d15715593af78fbdcad2b3660fb41cd4e4f71a4914086773ff87dba7bef
Deleted: sha256:297ab1e308eb83de323d81ae3c7ecba43f8bc9c349d89e91a614836d3e026169
Deleted: sha256:94bbb109593e34dc2640f7c37131343ecfeb6e6b419408223aeb744dd7c6c3bd
Deleted: sha256:9579645b6ba32a041286fba8f4ab6e27978ed86ddacf511efd764673282d8912
Deleted: sha256:fa542efa863ef840d258f8347951053e6fbdd328ef709d00c27fbc62d0212bf2
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:5b2ae7f2f6236e19e76398e7d6b65d13a5540b9333b4573697f77fc495e4b4a9
Associated tags:
- 20200305-060328
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-060328
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-060328].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:5b2ae7f2f6236e19e76398e7d6b65d13a5540b9333b4573697f77fc495e4b4a9].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 18m 15s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/c7yoauu2nxzrk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2594
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2594/display/redirect?page=changes>
Changes:
[filiperegadas] Add BigQuery useAvroLogicalTypes option
[filiperegadas] fixup! Add BigQuery useAvroLogicalTypes option
------------------------------------------
[...truncated 445.82 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 05:34:56 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 05:34:56 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 05:34:56 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 3dd112506e3227a1955202603b922777cfe5be0ca13fcda58576ec6ea82370bf. Logs:
2020/03/05 05:34:55 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583386486431810277" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583386486431810277-jenkins-0305053446-60da9cff" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583386486431810277_a36b6fe7-183b-48b8-9112-935932456ea1/MANIFEST" logging_endpoint:<url:"localhost:44043" > artifact_endpoint:<url:"localhost:34323" > control_endpoint:<url:"localhost:45125" >
2020/03/05 05:34:55 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:36497
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583386496 nanos:28109328 > message:"forcing DataChannel[localhost:37077] reconnection on port {localhost:37077} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 05:34:56 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-052939"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 3dd112506e3227a1955202603b922777cfe5be0ca13fcda58576ec6ea82370bf'. stderr: Error: No such container: 3dd112506e3227a1955202603b922777cfe5be0ca13fcda58576ec6ea82370bf
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583386486431810277_a36b6fe7-183b-48b8-9112-935932456ea1/MANIFEST
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583386486431810277_a36b6fe7-183b-48b8-9112-935932456ea1/MANIFEST -> 1 artifacts
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 05:34:58 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:44765 ...
20/03/05 05:34:58 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:46273 ...
20/03/05 05:34:58 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:39749 ...
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 05:34:58 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2655194 ns
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-052939"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 05:34:58 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 05:34:58 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container dd86b1033be2239411d4559c443d235f6f5b77d5c92ef64a3588ad6872ba642e. Logs:
2020/03/05 05:34:58 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583386486431810277" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583386486431810277-jenkins-0305053446-60da9cff" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583386486431810277_a36b6fe7-183b-48b8-9112-935932456ea1/MANIFEST" logging_endpoint:<url:"localhost:46273" > artifact_endpoint:<url:"localhost:46487" > control_endpoint:<url:"localhost:44765" >
2020/03/05 05:34:58 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:39721
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583386498 nanos:381912592 > message:"forcing DataChannel[localhost:39749] reconnection on port {localhost:39749} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 05:34:59 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-052939"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm dd86b1033be2239411d4559c443d235f6f5b77d5c92ef64a3588ad6872ba642e'. stderr: Error: No such container: dd86b1033be2239411d4559c443d235f6f5b77d5c92ef64a3588ad6872ba642e
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 05:34:59 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583386486431810277-jenkins-0305053446-60da9cff_5b9534aa-0d4a-4d97-8331-298b40171d24 finished.
20/03/05 05:34:59 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583386486431810277_a36b6fe7-183b-48b8-9112-935932456ea1/MANIFEST has 1 artifact locations
20/03/05 05:34:59 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583386486431810277_a36b6fe7-183b-48b8-9112-935932456ea1/
2020/03/05 05:34:59 Job state: DONE
2020/03/05 05:34:59 Test flatten:flatten completed
2020/03/05 05:34:59 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-052939" (must force) - container 5252f7406d52 is using its referenced image 84a5f55207d0
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:13be692806279f179d08db2e962eca4a09711bb3d412877cc4cb874be6362e09
Associated tags:
- 20200305-052939
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-052939
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-052939].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:13be692806279f179d08db2e962eca4a09711bb3d412877cc4cb874be6362e09].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
>>> FAILURE
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 11s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/4pnkv2xi4z6ec
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2593
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2593/display/redirect?page=changes>
Changes:
[github] Update container image tags used by Dataflow runner for Beam master
------------------------------------------
[...truncated 445.24 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 03:11:00 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 03:11:00 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 03:11:00 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 79d90cfd76a1addc7dd86d17212b73a419b94824b2e3143376121c326e175243. Logs:
2020/03/05 03:10:59 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583377849438179416" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583377849438179416-jenkins-0305031050-a7e53521" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583377849438179416_8ec98c9d-1d37-4d4d-9377-eec70082cc76/MANIFEST" logging_endpoint:<url:"localhost:44955" > artifact_endpoint:<url:"localhost:46525" > control_endpoint:<url:"localhost:42461" >
2020/03/05 03:10:59 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:39223
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583377860 nanos:223372243 > message:"forcing DataChannel[localhost:40511] reconnection on port {localhost:40511} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 03:11:01 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-030452"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 79d90cfd76a1addc7dd86d17212b73a419b94824b2e3143376121c326e175243'. stderr: Error: No such container: 79d90cfd76a1addc7dd86d17212b73a419b94824b2e3143376121c326e175243
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583377849438179416_8ec98c9d-1d37-4d4d-9377-eec70082cc76/MANIFEST
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583377849438179416_8ec98c9d-1d37-4d4d-9377-eec70082cc76/MANIFEST -> 1 artifacts
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 03:11:02 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:46517 ...
20/03/05 03:11:02 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:46205 ...
20/03/05 03:11:02 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:42319 ...
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 03:11:02 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 3774658 ns
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-030452"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 03:11:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 03:11:02 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 42264de210a47882e2c65719c26d2cebca388ac9351a120fd40e3a6e87aa7418. Logs:
2020/03/05 03:11:02 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583377849438179416" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583377849438179416-jenkins-0305031050-a7e53521" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583377849438179416_8ec98c9d-1d37-4d4d-9377-eec70082cc76/MANIFEST" logging_endpoint:<url:"localhost:46205" > artifact_endpoint:<url:"localhost:46399" > control_endpoint:<url:"localhost:46517" >
2020/03/05 03:11:02 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:46711
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583377862 nanos:684640803 > message:"forcing DataChannel[localhost:42319] reconnection on port {localhost:42319} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 03:11:03 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-030452"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 42264de210a47882e2c65719c26d2cebca388ac9351a120fd40e3a6e87aa7418'. stderr: Error: No such container: 42264de210a47882e2c65719c26d2cebca388ac9351a120fd40e3a6e87aa7418
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 03:11:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583377849438179416-jenkins-0305031050-a7e53521_4fb8eff6-01f1-41b3-8ef8-665f35ee26a3 finished.
20/03/05 03:11:03 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583377849438179416_8ec98c9d-1d37-4d4d-9377-eec70082cc76/MANIFEST has 1 artifact locations
20/03/05 03:11:03 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583377849438179416_8ec98c9d-1d37-4d4d-9377-eec70082cc76/
2020/03/05 03:11:03 Job state: DONE
2020/03/05 03:11:03 Test flatten:flatten completed
2020/03/05 03:11:03 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-030452" (must force) - container 642e379b3a8e is using its referenced image d094bbef388a
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:4feaa0587a85320ce864688a6377101ebcdbd2fd3f7492e38c0ca1507545b8cb
Associated tags:
- 20200305-030452
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-030452
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-030452].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:4feaa0587a85320ce864688a6377101ebcdbd2fd3f7492e38c0ca1507545b8cb].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 37s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/jbjax6tilr7ji
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org