You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/03/05 11:33:45 UTC
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2598
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2598/display/redirect?page=changes>
Changes:
[github] Add integration test for AnnotateVideoWithContext transform (#10986)
------------------------------------------
[...truncated 446.56 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 11:33:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 11:33:37 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 11:33:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c48277b856e7ed286548f439a8cfc78a9f1cd4492d75ed4c1f6374f34788d8d4. Logs:
2020/03/05 11:33:37 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583407984321584619" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583407984321584619-jenkins-0305113304-e7d372b3" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST" logging_endpoint:<url:"localhost:41705" > artifact_endpoint:<url:"localhost:39849" > control_endpoint:<url:"localhost:43203" >
2020/03/05 11:33:37 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:34507
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583408017 nanos:376491270 > message:"forcing DataChannel[localhost:35103] reconnection on port {localhost:35103} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 11:33:38 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm c48277b856e7ed286548f439a8cfc78a9f1cd4492d75ed4c1f6374f34788d8d4'. stderr: Error: No such container: c48277b856e7ed286548f439a8cfc78a9f1cd4492d75ed4c1f6374f34788d8d4
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST -> 1 artifacts
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41237 ...
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33965 ...
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41311 ...
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 11:33:39 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2822291 ns
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 11:33:39 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 11:33:39 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container caaf1004eed076999efd1efe95788da2d5fab2b33500fee7b1cbe2b353c50abe. Logs:
2020/03/05 11:33:39 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583407984321584619" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583407984321584619-jenkins-0305113304-e7d372b3" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST" logging_endpoint:<url:"localhost:33965" > artifact_endpoint:<url:"localhost:45949" > control_endpoint:<url:"localhost:41237" >
2020/03/05 11:33:39 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:38397
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583408019 nanos:658509719 > message:"forcing DataChannel[localhost:41311] reconnection on port {localhost:41311} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 11:33:40 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm caaf1004eed076999efd1efe95788da2d5fab2b33500fee7b1cbe2b353c50abe'. stderr: Error: No such container: caaf1004eed076999efd1efe95788da2d5fab2b33500fee7b1cbe2b353c50abe
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 11:33:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583407984321584619-jenkins-0305113304-e7d372b3_c75f545a-e720-40d6-84ec-8f4d5ba2e8f3 finished.
20/03/05 11:33:40 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/MANIFEST has 1 artifact locations
20/03/05 11:33:40 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583407984321584619_98c42372-0da7-4ed2-acb8-91362a403211/
2020/03/05 11:33:40 Job state: DONE
2020/03/05 11:33:40 Test flatten:flatten completed
2020/03/05 11:33:40 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704" (must force) - container 6840f938b5cf is using its referenced image 61da195ceb6b
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:ea80e2a5495966600332a5041b1ea4d78b35ca5062aa10c5193e41ddfd542125
Associated tags:
- 20200305-112704
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-112704].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:ea80e2a5495966600332a5041b1ea4d78b35ca5062aa10c5193e41ddfd542125].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
>>> FAILURE
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 40s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/b76i3flszgtyw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Go_VR_Spark #2608
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2608/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2607
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2607/display/redirect?page=changes>
Changes:
[chuck.yang] Use Avro format for file loads to BigQuery
------------------------------------------
[...truncated 445.58 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 01:11:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 01:11:50 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 01:11:50 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2b6e5cb7585549ae4ee5dbd41ab807bc6ba468af6f6fdb246c7767b361fb351f. Logs:
2020/03/06 01:11:50 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583457097396906625" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583457097396906625-jenkins-0306011138-c9fb6c68" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST" logging_endpoint:<url:"localhost:34359" > artifact_endpoint:<url:"localhost:36151" > control_endpoint:<url:"localhost:46819" >
2020/03/06 01:11:50 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:34503
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583457110 nanos:650448304 > message:"forcing DataChannel[localhost:34141] reconnection on port {localhost:34141} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/06 01:11:51 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 2b6e5cb7585549ae4ee5dbd41ab807bc6ba468af6f6fdb246c7767b361fb351f'. stderr: Error: No such container: 2b6e5cb7585549ae4ee5dbd41ab807bc6ba468af6f6fdb246c7767b361fb351f
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 01:11:52 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST
20/03/06 01:11:52 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST -> 1 artifacts
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:32811 ...
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:42175 ...
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35227 ...
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/06 01:11:53 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 5054053 ns
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 01:11:53 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 01:11:53 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6865b1c5da5f6d598cf545a2c7927c66da7a4a3d488ddfcb75c06a1f200264fa. Logs:
2020/03/06 01:11:52 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583457097396906625" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583457097396906625-jenkins-0306011138-c9fb6c68" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST" logging_endpoint:<url:"localhost:42175" > artifact_endpoint:<url:"localhost:41947" > control_endpoint:<url:"localhost:32811" >
2020/03/06 01:11:52 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:42917
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583457113 nanos:527848417 > message:"forcing DataChannel[localhost:35227] reconnection on port {localhost:35227} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/06 01:11:54 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 6865b1c5da5f6d598cf545a2c7927c66da7a4a3d488ddfcb75c06a1f200264fa'. stderr: Error: No such container: 6865b1c5da5f6d598cf545a2c7927c66da7a4a3d488ddfcb75c06a1f200264fa
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 01:11:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583457097396906625-jenkins-0306011138-c9fb6c68_dd5bfe87-161d-4b82-843a-100255f002c8 finished.
20/03/06 01:11:54 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/MANIFEST has 1 artifact locations
20/03/06 01:11:54 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583457097396906625_98877a92-6020-4694-9c40-0133410d4345/
2020/03/06 01:11:54 Job state: DONE
2020/03/06 01:11:54 Test flatten:flatten completed
2020/03/06 01:11:54 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631" (must force) - container 9c27e24ec8cd is using its referenced image f3a86203f6d7
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Failed to remove container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:57db3db7e6817618a03864efd77ad8c34385cc326246e77ee77c6fccf631b28c
Associated tags:
- 20200306-010631
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-010631].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:57db3db7e6817618a03864efd77ad8c34385cc326246e77ee77c6fccf631b28c].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
>>> FAILURE
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 30s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/jk63ot3yswqcy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2606
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2606/display/redirect>
Changes:
------------------------------------------
[...truncated 447.62 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 00:09:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 00:09:15 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 00:09:15 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2d0b83d6cc58e81f490748b911358558d7a3218fedcd434610fb390cb7215f43. Logs:
2020/03/06 00:09:15 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583453337143350768" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583453337143350768-jenkins-0306000857-44df8858" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST" logging_endpoint:<url:"localhost:40443" > artifact_endpoint:<url:"localhost:36355" > control_endpoint:<url:"localhost:32931" >
2020/03/06 00:09:15 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:44975
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583453355 nanos:606111561 > message:"forcing DataChannel[localhost:37089] reconnection on port {localhost:37089} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/06 00:09:16 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 2d0b83d6cc58e81f490748b911358558d7a3218fedcd434610fb390cb7215f43'. stderr: Error: No such container: 2d0b83d6cc58e81f490748b911358558d7a3218fedcd434610fb390cb7215f43
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST -> 1 artifacts
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:39315 ...
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41613 ...
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35527 ...
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/06 00:09:20 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 3193227 ns
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/06 00:09:20 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/06 00:09:20 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c9a67ed6a767359909d62bfd7989f5c71d01d54966a5a02b878f4e5c12621f4a. Logs:
2020/03/06 00:09:20 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583453337143350768" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583453337143350768-jenkins-0306000857-44df8858" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST" logging_endpoint:<url:"localhost:41613" > artifact_endpoint:<url:"localhost:37715" > control_endpoint:<url:"localhost:39315" >
2020/03/06 00:09:20 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:42257
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583453360 nanos:342237218 > message:"forcing DataChannel[localhost:35527] reconnection on port {localhost:35527} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/06 00:09:21 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm c9a67ed6a767359909d62bfd7989f5c71d01d54966a5a02b878f4e5c12621f4a'. stderr: Error: No such container: c9a67ed6a767359909d62bfd7989f5c71d01d54966a5a02b878f4e5c12621f4a
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/06 00:09:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583453337143350768-jenkins-0306000857-44df8858_0618ec9b-3083-48e9-b12b-6c3de7a5dc32 finished.
20/03/06 00:09:21 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/MANIFEST has 1 artifact locations
20/03/06 00:09:21 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583453337143350768_8e5e55b5-0a12-4065-8e06-551def0d0574/
2020/03/06 00:09:21 Job state: DONE
2020/03/06 00:09:21 Test flatten:flatten completed
2020/03/06 00:09:21 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233" (must force) - container 939e11895d10 is using its referenced image 5ad656894196
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:5b10dc23af8408ed8bca9af7d40f7f403ff72a9e3dbd65038754d1a7957b04cc
Associated tags:
- 20200306-000233
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200306-000233].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:5b10dc23af8408ed8bca9af7d40f7f403ff72a9e3dbd65038754d1a7957b04cc].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 1s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/iwz7xm3wrn23g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2605
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2605/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-9450] Update www.apache.org/dist/ links to downloads.apache.org
[iemejia] [BEAM-9450] Convert links available via https to use https
------------------------------------------
[...truncated 164.74 KB...]
>
pcollections: <
key: "n3"
value: <
unique_name: "n3"
coder_id: "c0"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n4"
value: <
unique_name: "n4"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n5"
value: <
unique_name: "n5"
coder_id: "c0"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n6"
value: <
unique_name: "n6"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n7"
value: <
unique_name: "n7"
coder_id: "c3"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n8"
value: <
unique_name: "n8"
coder_id: "c4"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
pcollections: <
key: "n9"
value: <
unique_name: "n9"
coder_id: "c6"
is_bounded: BOUNDED
windowing_strategy_id: "w0"
>
>
windowing_strategies: <
key: "w0"
value: <
window_fn: <
urn: "beam:window_fn:global_windows:v1"
>
merge_status: NON_MERGING
window_coder_id: "c1"
trigger: <
default: <
>
>
accumulation_mode: DISCARDING
output_time: END_OF_WINDOW
closing_behavior: EMIT_IF_NONEMPTY
OnTimeBehavior: FIRE_ALWAYS
environment_id: "go"
>
>
coders: <
key: "c0"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c1"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:go:coder:custom:v1"
payload: "Cgd2YXJpbnR6EgIIAhqFAQpxZ2l0aHViLmNvbS9hcGFjaGUvYmVhbS9zZGtzL2dvL3Rlc3QvdmVuZG9yL2dpdGh1Yi5jb20vYXBhY2hlL2JlYW0vc2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvY29kZXJ4LmVuY1ZhckludFoSEAgWIgQIGUAPKgYIFBICCAgikQEKcWdpdGh1Yi5jb20vYXBhY2hlL2JlYW0vc2Rrcy9nby90ZXN0L3ZlbmRvci9naXRodWIuY29tL2FwYWNoZS9iZWFtL3Nka3MvZ28vcGtnL2JlYW0vY29yZS9ydW50aW1lL2NvZGVyeC5kZWNWYXJJbnRaEhwIFiIECBlAAyIGCBQSAggIKgQIGUAPKgQIGUAB"
>
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c3"
component_coder_ids: "c3"
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c3"
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c3"
component_coder_ids: "c5"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
>
>
>
root_transform_ids: "e5"
root_transform_ids: "e3"
root_transform_ids: "e4"
root_transform_ids: "e1"
root_transform_ids: "e2"
root_transform_ids: "e6"
root_transform_ids: "e7"
root_transform_ids: "s1"
2020/03/05 22:02:14 Test flatten:flatten failed: connecting to job service:
failed to dial server at localhost:60151
caused by:
context deadline exceeded
2020/03/05 22:02:14 Result: 7 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:67223f1419626f15845dfcdaa0357ecf30ef483fcb31d9852c6b7cdee15e1d68
Deleted: sha256:040b4978b5b62938938d75551d31ef6b4f506cc96efb7b253aca74c560568f09
Deleted: sha256:f2ccdb331bad0513f6794488dcf1cdd9e7b0ad18815f2f326584235545e836c5
Deleted: sha256:71af76733efbc7a3d399c36900a4f21f4824beee3c414adcea4fe1534e56960f
Deleted: sha256:baa68cea33770e5564ec9a72aaad9aed3053a0d9c58da57a49e8dff5a62f3e7f
Deleted: sha256:0d3f9e843e0718adbd2f52e1cd48ca4a100a7f3e50e6a56b26c799c40ffb36d4
Deleted: sha256:b16063a781f21e2a03d01bffb7c8315d8870fe9f71e06ed5e7c340bda2db8a99
Deleted: sha256:0e8ed0424902cbb715b9f0cb54c3ff72da9965680b2fcdb961780b5e274282d8
Deleted: sha256:ddd28c44250c585cf7df4c03c2736efdd021d7d2dd156bf5fbeeda98532dcbe9
Deleted: sha256:8e41a512105114d0e5d1ca1b8d80087ef1c8a41e04ef356d85167af2ae53685f
Deleted: sha256:d8f086eeeb6aacd554b2261f7cf9e57e71bc0b6209690aee8d6ea37a93456fae
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:67223f1419626f15845dfcdaa0357ecf30ef483fcb31d9852c6b7cdee15e1d68
Associated tags:
- 20200305-214711
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-214711].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:67223f1419626f15845dfcdaa0357ecf30ef483fcb31d9852c6b7cdee15e1d68].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 18m 27s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/rg25dyyy4mm22
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2604
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2604/display/redirect?page=changes>
Changes:
[git] Remove optionality and add sensible defaults to PubsubIO builders.
------------------------------------------
[...truncated 445.84 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 21:09:25 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 21:09:25 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 21:09:25 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 585e325420f01e1ce799b163d22d8439f979f7f2d6712de4e9f3af2b0671fe5d. Logs:
2020/03/05 21:09:25 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583442557218548443" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583442557218548443-jenkins-0305210917-8b42e2d" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST" logging_endpoint:<url:"localhost:35671" > artifact_endpoint:<url:"localhost:32937" > control_endpoint:<url:"localhost:46171" >
2020/03/05 21:09:25 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:39699
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583442565 nanos:842808066 > message:"forcing DataChannel[localhost:46477] reconnection on port {localhost:46477} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 21:09:26 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 585e325420f01e1ce799b163d22d8439f979f7f2d6712de4e9f3af2b0671fe5d'. stderr: Error: No such container: 585e325420f01e1ce799b163d22d8439f979f7f2d6712de4e9f3af2b0671fe5d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 21:09:27 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST
20/03/05 21:09:27 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST -> 1 artifacts
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35009 ...
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:34081 ...
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41901 ...
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 21:09:28 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2800435 ns
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 21:09:28 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 903b8f5367c2cdc1034e6247bc2ed159a2222d45824c393352d1c8c990679a87. Logs:
2020/03/05 21:09:27 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583442557218548443" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583442557218548443-jenkins-0305210917-8b42e2d" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST" logging_endpoint:<url:"localhost:34081" > artifact_endpoint:<url:"localhost:42371" > control_endpoint:<url:"localhost:35009" >
2020/03/05 21:09:27 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:33497
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583442568 nanos:116743726 > message:"forcing DataChannel[localhost:41901] reconnection on port {localhost:41901} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 21:09:28 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 903b8f5367c2cdc1034e6247bc2ed159a2222d45824c393352d1c8c990679a87'. stderr: Error: No such container: 903b8f5367c2cdc1034e6247bc2ed159a2222d45824c393352d1c8c990679a87
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 21:09:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583442557218548443-jenkins-0305210917-8b42e2d_722ca76d-5040-4a4a-9188-0d0c67db4f7b finished.
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/MANIFEST has 1 artifact locations
20/03/05 21:09:28 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583442557218548443_1b938595-af5e-4e78-b021-2452f5f1c9e0/
2020/03/05 21:09:28 Job state: DONE
2020/03/05 21:09:28 Test flatten:flatten completed
2020/03/05 21:09:28 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335" (must force) - container dceef77112d2 is using its referenced image 8cdd34626838
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:c192848e6b16c5452c2388cc2e4b1fd73fcc297ee4f54581c4e285f1ecb9c1e7
Associated tags:
- 20200305-210335
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-210335].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:c192848e6b16c5452c2388cc2e4b1fd73fcc297ee4f54581c4e285f1ecb9c1e7].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
>>> FAILURE
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 57s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/i2kx5blg7wkai
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2603
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2603/display/redirect?page=changes>
Changes:
[jkai] [Hotfix] fix rabbitmp spotless check
------------------------------------------
[...truncated 447.45 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 20:19:48 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 20:19:48 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 20:19:48 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container af62669adcb526746847b11625ad132e2048e13b517882573f370b232f65ec41. Logs:
2020/03/05 20:19:47 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583439579607091859" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583439579607091859-jenkins-0305201940-e221b732" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST" logging_endpoint:<url:"localhost:33023" > artifact_endpoint:<url:"localhost:40233" > control_endpoint:<url:"localhost:43991" >
2020/03/05 20:19:47 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:33025
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583439588 nanos:235425853 > message:"forcing DataChannel[localhost:39719] reconnection on port {localhost:39719} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 20:19:48 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm af62669adcb526746847b11625ad132e2048e13b517882573f370b232f65ec41'. stderr: Error: No such container: af62669adcb526746847b11625ad132e2048e13b517882573f370b232f65ec41
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST -> 1 artifacts
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35697 ...
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33571 ...
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:41689 ...
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 20:19:50 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2826109 ns
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 20:19:50 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 20:19:50 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 98a4c3bf32e85cf35e9d536ca575f9211002c5935d99cd3d7c5be46fedc055b3. Logs:
2020/03/05 20:19:50 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583439579607091859" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583439579607091859-jenkins-0305201940-e221b732" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST" logging_endpoint:<url:"localhost:33571" > artifact_endpoint:<url:"localhost:40965" > control_endpoint:<url:"localhost:35697" >
2020/03/05 20:19:50 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:39935
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583439590 nanos:321899958 > message:"forcing DataChannel[localhost:41689] reconnection on port {localhost:41689} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 20:19:51 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 98a4c3bf32e85cf35e9d536ca575f9211002c5935d99cd3d7c5be46fedc055b3'. stderr: Error: No such container: 98a4c3bf32e85cf35e9d536ca575f9211002c5935d99cd3d7c5be46fedc055b3
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 20:19:51 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583439579607091859-jenkins-0305201940-e221b732_c6a6be5d-26fc-48e4-9ccc-540606bfb7da finished.
20/03/05 20:19:51 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/MANIFEST has 1 artifact locations
20/03/05 20:19:51 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583439579607091859_f2971108-70ad-4172-84a4-59dddb33facd/
2020/03/05 20:19:51 Job state: DONE
2020/03/05 20:19:51 Test flatten:flatten completed
2020/03/05 20:19:51 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434" (must force) - container 0e063d24ebad is using its referenced image 8ab8d3c9143d
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:426fd7ae8af07c913cfdaea8d1edbecc6a76e2fc9b184a6386c9d4289d8ace57
Associated tags:
- 20200305-201434
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-201434].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:426fd7ae8af07c913cfdaea8d1edbecc6a76e2fc9b184a6386c9d4289d8ace57].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
>>> FAILURE
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 20s
55 actionable tasks: 42 executed, 13 from cache
Publishing build scan...
https://gradle.com/s/ookdqbxna7fns
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2602
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2602/display/redirect?page=changes>
Changes:
[amaliujia] [BEAM-9288] Not bundle conscrypt in gRPC vendor in META-INF/
[lcwik] [BEAM-9452] Update classgraph to latest version to resolve windows
------------------------------------------
[...truncated 458.61 KB...]
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:125)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:159)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:201)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:185)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 32 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error response from daemon: Cannot kill container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d: No such container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:148)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:186)
... 42 more
20/03/05 18:57:22 ERROR org.apache.spark.scheduler.TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
20/03/05 18:57:22 ERROR org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation: Error during job invocation go0job0701583434627048740453-jenkins-0305185707-de3c0e7c_ddcb107c-8f98-4ae0-ad9f-747169145594.
org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:71)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:102)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:90)
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:139)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:333)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:321)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:251)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.getStageBundleFactory(DefaultExecutableStageContext.java:38)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingExecutableStageContextFactory.java:198)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:125)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
... 3 more
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:159)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:201)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:185)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 32 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error response from daemon: Cannot kill container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d: No such container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:148)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:186)
... 42 more
2020/03/05 18:57:22 (): org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:71)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:102)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:90)
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:139)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:333)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:321)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:251)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.getStageBundleFactory(DefaultExecutableStageContext.java:38)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingExecutableStageContextFactory.java:198)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:125)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
... 3 more
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:159)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:201)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:185)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 32 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error response from daemon: Cannot kill container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d: No such container: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:148)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:186)
... 42 more
2020/03/05 18:57:22 (): java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d'. stderr: Error: No such object: e7102275c7a845e5fa0a1b9960a9af5e6f7f144aa43581340a6bb91e2dab7d9d
20/03/05 18:57:22 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583434627048740453_4cf6a1ce-0f09-4780-b451-04932fc37444/MANIFEST has 1 artifact locations
20/03/05 18:57:22 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583434627048740453_4cf6a1ce-0f09-4780-b451-04932fc37444/
2020/03/05 18:57:22 Job state: FAILED
2020/03/05 18:57:22 Test flatten:flatten failed: job go0job0701583434627048740453-jenkins-0305185707-de3c0e7c_ddcb107c-8f98-4ae0-ad9f-747169145594 failed
2020/03/05 18:57:22 Result: 2 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-185143" (must force) - container b45cd7f62242 is using its referenced image 965c9ef327b8
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:9714b326adea54e228bd5d9aac920f7526a0fca99fc5c43a0428db0f0dce6e7c
Associated tags:
- 20200305-185143
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-185143
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-185143].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:9714b326adea54e228bd5d9aac920f7526a0fca99fc5c43a0428db0f0dce6e7c].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
>>> FAILURE
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 27s
55 actionable tasks: 54 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/xlbpqf6ivs2ra
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2601
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2601/display/redirect>
Changes:
------------------------------------------
[...truncated 447.21 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 18:11:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 18:11:14 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 18:11:14 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e45e0c28ddc26d142d28a2dc4cdda0ddb4bd9e979dea2556df0a3e8bd6e26d14. Logs:
2020/03/05 18:11:14 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583431865548803456" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583431865548803456-jenkins-0305181105-40e21032" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST" logging_endpoint:<url:"localhost:45727" > artifact_endpoint:<url:"localhost:33779" > control_endpoint:<url:"localhost:35607" >
2020/03/05 18:11:14 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:39599
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583431874 nanos:464957596 > message:"forcing DataChannel[localhost:41825] reconnection on port {localhost:41825} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 18:11:15 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm e45e0c28ddc26d142d28a2dc4cdda0ddb4bd9e979dea2556df0a3e8bd6e26d14'. stderr: Error: No such container: e45e0c28ddc26d142d28a2dc4cdda0ddb4bd9e979dea2556df0a3e8bd6e26d14
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST -> 1 artifacts
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:34145 ...
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33233 ...
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:42803 ...
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 18:11:16 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2708295 ns
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 18:11:16 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 18:11:16 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 889fc62c7dec204013ca12a5c62eae97a604908aa5c67e92af1f8e38f804644a. Logs:
2020/03/05 18:11:16 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583431865548803456" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583431865548803456-jenkins-0305181105-40e21032" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST" logging_endpoint:<url:"localhost:33233" > artifact_endpoint:<url:"localhost:36191" > control_endpoint:<url:"localhost:34145" >
2020/03/05 18:11:16 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:33395
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583431876 nanos:752070044 > message:"forcing DataChannel[localhost:42803] reconnection on port {localhost:42803} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 18:11:17 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 889fc62c7dec204013ca12a5c62eae97a604908aa5c67e92af1f8e38f804644a'. stderr: Error: No such container: 889fc62c7dec204013ca12a5c62eae97a604908aa5c67e92af1f8e38f804644a
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 18:11:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583431865548803456-jenkins-0305181105-40e21032_5cdd34f9-c288-4838-af15-f67ebca1ad44 finished.
20/03/05 18:11:17 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/MANIFEST has 1 artifact locations
20/03/05 18:11:17 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583431865548803456_672d2b55-ca17-4359-8c75-455155e8f856/
2020/03/05 18:11:17 Job state: DONE
2020/03/05 18:11:17 Test flatten:flatten completed
2020/03/05 18:11:17 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601" (must force) - container f5f1fb296fb6 is using its referenced image 4f396679cc4e
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:3915d59fa31148c53ed69a293aa9e8ee88ab55bfba6850a3a674a60ba331e87a
Associated tags:
- 20200305-180601
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-180601].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:3915d59fa31148c53ed69a293aa9e8ee88ab55bfba6850a3a674a60ba331e87a].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 5s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/qynppn4tgpfj6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2600
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2600/display/redirect?page=changes>
Changes:
[hktang] [BEAM-9453] Changed new string creation to use StandardCharsets.UTF_8
------------------------------------------
[...truncated 446.50 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 17:45:35 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 17:45:35 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 17:45:35 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1c439a64d455133af9ef8ce2e31c1d3a3cf9769deb99e25862c7828c3773bd5c. Logs:
2020/03/05 17:45:35 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583430326975248778" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583430326975248778-jenkins-0305174527-7fd4806f" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST" logging_endpoint:<url:"localhost:40437" > artifact_endpoint:<url:"localhost:35343" > control_endpoint:<url:"localhost:42231" >
2020/03/05 17:45:35 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:42579
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583430335 nanos:700689327 > message:"forcing DataChannel[localhost:44581] reconnection on port {localhost:44581} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 17:45:36 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 1c439a64d455133af9ef8ce2e31c1d3a3cf9769deb99e25862c7828c3773bd5c'. stderr: Error: No such container: 1c439a64d455133af9ef8ce2e31c1d3a3cf9769deb99e25862c7828c3773bd5c
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST -> 1 artifacts
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:34479 ...
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:32807 ...
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:33041 ...
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 17:45:37 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 2628866 ns
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 17:45:37 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 17:45:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f17032c2be6020422e39011c834faa6caeef2f207ec3489b1c9be730c3a53fd5. Logs:
2020/03/05 17:45:37 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583430326975248778" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583430326975248778-jenkins-0305174527-7fd4806f" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST" logging_endpoint:<url:"localhost:32807" > artifact_endpoint:<url:"localhost:39973" > control_endpoint:<url:"localhost:34479" >
2020/03/05 17:45:37 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:43559
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583430337 nanos:933836329 > message:"forcing DataChannel[localhost:33041] reconnection on port {localhost:33041} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 17:45:38 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm f17032c2be6020422e39011c834faa6caeef2f207ec3489b1c9be730c3a53fd5'. stderr: Error: No such container: f17032c2be6020422e39011c834faa6caeef2f207ec3489b1c9be730c3a53fd5
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 17:45:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583430326975248778-jenkins-0305174527-7fd4806f_1a79ba9a-1a6e-447d-86ed-ed062ab7cbb3 finished.
20/03/05 17:45:38 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/MANIFEST has 1 artifact locations
20/03/05 17:45:38 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583430326975248778_b336433b-852c-41d5-b74f-10b58884ddcc/
2020/03/05 17:45:38 Job state: DONE
2020/03/05 17:45:38 Test flatten:flatten completed
2020/03/05 17:45:38 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001" (must force) - container 570faffde618 is using its referenced image fb94576c7e68
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:38ea694cadd831c30d3ebac35986e69a074ecc3bbffa60f7c1141f61594a1952
Associated tags:
- 20200305-174001
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-174001].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:38ea694cadd831c30d3ebac35986e69a074ecc3bbffa60f7c1141f61594a1952].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 51s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/mxepcgok5ip66
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #2599
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/2599/display/redirect>
Changes:
------------------------------------------
[...truncated 445.67 KB...]
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 12:07:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 12:07:42 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 12:07:42 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9af00c6b81c7db2daa8c1784b67f7641d730718fd41e0e9f95c602d7dd154a07. Logs:
2020/03/05 12:07:42 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583410048110577803" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583410048110577803-jenkins-0305120728-71b9f994" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST" logging_endpoint:<url:"localhost:38005" > artifact_endpoint:<url:"localhost:34527" > control_endpoint:<url:"localhost:39805" >
2020/03/05 12:07:42 Initializing Go harness: /opt/apache/beam/boot --id=23-1 --provision_endpoint=localhost:37661
Worker exited successfully!
Failed to send message: EOF
severity:WARN timestamp:<seconds:1583410062 nanos:622736736 > message:"forcing DataChannel[localhost:41721] reconnection on port {localhost:41721} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: EOF. Retrying in 5 sec ...
20/03/05 12:07:43 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 9af00c6b81c7db2daa8c1784b67f7641d730718fd41e0e9f95c602d7dd154a07'. stderr: Error: No such container: 9af00c6b81c7db2daa8c1784b67f7641d730718fd41e0e9f95c602d7dd154a07
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:105)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: GetManifest for /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST -> 1 artifacts
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 24-1
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:35383 ...
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:38253 ...
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:331: Connecting via grpc @ localhost:37151 ...
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
20/03/05 12:07:44 INFO <https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:246: DataSource: 1 elements in 3788293 ns
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
20/03/05 12:07:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
20/03/05 12:07:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
20/03/05 12:07:45 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1cc41f99352256425c0860364c14ff4438416b176f68b196b8a49b0627dcb831. Logs:
2020/03/05 12:07:44 Provision info:
pipeline_options:<fields:<key:"beam:option:app_name:v1" value:<string_value:"go-job-7-1583410048110577803" > > fields:<key:"beam:option:experiments:v1" value:<list_value:<values:<string_value:"beam_fn_api" > > > > fields:<key:"beam:option:go_options:v1" value:<struct_value:<fields:<key:"options" value:<struct_value:<fields:<key:"hooks" value:<string_value:"{}" > > > > > > > > fields:<key:"beam:option:job_name:v1" value:<string_value:"go0job0701583410048110577803-jenkins-0305120728-71b9f994" > > fields:<key:"beam:option:options_id:v1" value:<number_value:7 > > fields:<key:"beam:option:runner:v1" value:<null_value:NULL_VALUE > > fields:<key:"beam:option:spark_master:v1" value:<string_value:"local" > > > retrieval_token:"/tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST" logging_endpoint:<url:"localhost:38253" > artifact_endpoint:<url:"localhost:42449" > control_endpoint:<url:"localhost:35383" >
2020/03/05 12:07:44 Initializing Go harness: /opt/apache/beam/boot --id=24-1 --provision_endpoint=localhost:46181
Worker exited successfully!
Failed to send message: rpc error: code = Unavailable desc = transport is closing
severity:WARN timestamp:<seconds:1583410064 nanos:981937259 > message:"forcing DataChannel[localhost:37151] reconnection on port {localhost:37151} due to rpc error: code = Canceled desc = Multiplexer hanging up" instruction_id:"2" log_location:"<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/datamgr.go>:119" Remote logging failed: rpc error: code = Unavailable desc = transport is closing. Retrying in 5 sec ...
20/03/05 12:07:45 WARN org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Error cleaning up servers urn: "beam:env:docker:v1"
payload: "\nAus.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
java.io.IOException: Received exit code 1 for command 'docker rm 1cc41f99352256425c0860364c14ff4438416b176f68b196b8a49b0627dcb831'. stderr: Error: No such container: 1cc41f99352256425c0860364c14ff4438416b176f68b196b8a49b0627dcb831
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:479)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:494)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$1600(DefaultJobBundleFactory.java:432)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:169)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:259)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:132)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:44)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:204)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.$closeResource(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:189)
at org.apache.beam.runners.spark.translation.SparkExecutableStageFunction.call(SparkExecutableStageFunction.java:80)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(JavaRDDLike.scala:153)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:823)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
20/03/05 12:07:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0701583410048110577803-jenkins-0305120728-71b9f994_eb231148-1c04-4a0c-aac5-8a6bbd8eb074 finished.
20/03/05 12:07:45 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/MANIFEST has 1 artifact locations
20/03/05 12:07:45 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-artifact-staging/go-job-7-1583410048110577803_dbf6fac7-bd1e-4fa0-ae3c-507aaa07af35/
2020/03/05 12:07:45 Job state: DONE
2020/03/05 12:07:45 Test flatten:flatten completed
2020/03/05 12:07:45 Result: 1 tests failed
if [[ ! -z "$JOB_PORT" ]]; then
# Shut down the job server
kill %1 || echo "Failed to shut down job server"
fi
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
Error response from daemon: conflict: unable to remove repository reference "us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222" (must force) - container ea8f75f6c0d0 is using its referenced image 8d7199a204c5
Failed to remove container
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:59dffdefc146173c01312f6638b520404d911df8e970d22bbbb075b117933b78
Associated tags:
- 20200305-120222
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk:20200305-120222].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_go_sdk@sha256:59dffdefc146173c01312f6638b520404d911df8e970d22bbbb075b117933b78].
# Clean up tempdir
rm -rf $TMPDIR
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
>>> FAILURE
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 75
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 29s
55 actionable tasks: 41 executed, 14 from cache
Publishing build scan...
https://gradle.com/s/5qj4tgx5hynlw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org