You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/05/25 18:14:10 UTC
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4719
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4719/display/redirect?page=changes>
Changes:
[noreply] Update test_error_message_includes_stage test
[samuelw] [BEAM-12402] Optimize
[noreply] [BEAM-12284] Use the element coder when pickling beam.Create. (#14834)
------------------------------------------
[...truncated 564.51 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/25 18:14:01 Prepared job with id: go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2 and staging token: go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2
2021/05/25 18:14:01 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621966441225354577
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.go.
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.null.
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.
2021/05/25 18:14:04 Staged binary artifact with token:
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
21/05/25 18:14:04 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
2021/05/25 18:14:04 Submitted job: go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
2021/05/25 18:14:04 Job state: STOPPED
2021/05/25 18:14:04 Job state: STARTING
2021/05/25 18:14:04 Job state: RUNNING
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1 on Spark master local
21/05/25 18:14:04 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1: Pipeline translated successfully. Computing outputs
21/05/25 18:14:04 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37291 ...
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40033 ...
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36239 ...
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3740834 ns
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1945557 ns
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/928ef1ad3445a9ce5592ee07dc95461c6e5015c8be749976e9b76ce26d004659/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/25 18:14:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container cf7b389edf596769b82f9e93a0c3f0edd4be87c98a697478b4f6fe173ce89d35. Logs:
2021/05/25 18:14:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621966441222215423"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621966441222215423-jenkins-0525181404-ee8c4d02"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40033"} artifact_endpoint:{url:"localhost:41285"} control_endpoint:{url:"localhost:37291"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/928ef1ad3445a9ce5592ee07dc95461c6e5015c8be749976e9b76ce26d004659/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/25 18:14:05 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:46509
2021/05/25 18:14:05 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/25 18:14:05 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/25 18:14:05 control response channel closed
21/05/25 18:14:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1 finished.
2021/05/25 18:14:06 Job state: DONE
21/05/25 18:14:06 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
21/05/25 18:14:06 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
2021/05/25 18:14:06 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (28.44s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 28.460s
2021/05/25 18:14:08 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 44s
79 actionable tasks: 57 executed, 22 from cache
Publishing build scan...
https://gradle.com/s/leobk3lh47ipm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Go_VR_Spark #4751
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4751/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4750
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4750/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12281] Drop support for Flink 1.10
[Ismaël Mejía] [BEAM-12281] Update Flink Jenkins jobs to use Flink 1.12
------------------------------------------
[...truncated 545.66 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/02 12:11:05 Prepared job with id: go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813 and staging token: go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813
2021/06/02 12:11:05 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622635865887115713
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.go.
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.null.
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.
2021/06/02 12:11:08 Staged binary artifact with token:
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
21/06/02 12:11:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
2021/06/02 12:11:08 Submitted job: go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
2021/06/02 12:11:08 Job state: STOPPED
2021/06/02 12:11:08 Job state: STARTING
2021/06/02 12:11:08 Job state: RUNNING
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7 on Spark master local
21/06/02 12:11:08 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7: Pipeline translated successfully. Computing outputs
21/06/02 12:11:09 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35417 ...
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44973 ...
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36153 ...
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2828080 ns
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1552704 ns
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e0172f1adb46b13234df41f900e309ca543aa0fc456b98534e1a9d7a46240b88/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/02 12:11:10 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 8ecfabecf337a1a2b40b86aabed06600cea071b40368848d46e739ee2f14ab61. Logs:
2021/06/02 12:11:09 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622635865884473617"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622635865884473617-jenkins-0602121108-b9829dd5"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44973"} artifact_endpoint:{url:"localhost:34739"} control_endpoint:{url:"localhost:35417"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e0172f1adb46b13234df41f900e309ca543aa0fc456b98534e1a9d7a46240b88/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/02 12:11:09 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:36921
2021/06/02 12:11:10 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/02 12:11:10 control response channel closed
21/06/02 12:11:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7 finished.
2021/06/02 12:11:10 Job state: DONE
21/06/02 12:11:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
21/06/02 12:11:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
2021/06/02 12:11:10 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (45.42s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 45.438s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.020s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 45s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/veaqtn3rt5muo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4749
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4749/display/redirect?page=changes>
Changes:
[Brian Hulette] Add DataFrame API updates to CHANGES.md
------------------------------------------
[...truncated 545.97 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/02 06:12:58 Prepared job with id: go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617 and staging token: go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617
2021/06/02 06:12:58 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622614378322587806
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.go.
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.null.
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.
2021/06/02 06:13:00 Staged binary artifact with token:
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
21/06/02 06:13:00 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
2021/06/02 06:13:00 Submitted job: go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
2021/06/02 06:13:00 Job state: STOPPED
2021/06/02 06:13:00 Job state: STARTING
2021/06/02 06:13:00 Job state: RUNNING
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9 on Spark master local
21/06/02 06:13:00 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9: Pipeline translated successfully. Computing outputs
21/06/02 06:13:01 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45403 ...
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:41489 ...
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39363 ...
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2433466 ns
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1340047 ns
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/584d750791270944e481cc9948ee9d414e4d4bf9f5e44c170587553430ecc14b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/02 06:13:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container bbf5ffd687f32aaef8fa2f9407b92b0f09a1a3d4e31c46c347386649c514c5d5. Logs:
2021/06/02 06:13:02 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622614378320056534"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622614378320056534-jenkins-0602061300-152bfb7a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41489"} artifact_endpoint:{url:"localhost:43555"} control_endpoint:{url:"localhost:45403"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/584d750791270944e481cc9948ee9d414e4d4bf9f5e44c170587553430ecc14b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/02 06:13:02 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:42195
2021/06/02 06:13:02 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
2021/06/02 06:13:02 control response channel closed
Worker exited successfully!
21/06/02 06:13:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9 finished.
2021/06/02 06:13:03 Job state: DONE
21/06/02 06:13:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
21/06/02 06:13:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
2021/06/02 06:13:03 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.37s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.397s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 36s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/swok6bh6cfj7m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4748
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4748/display/redirect?page=changes>
Changes:
[anant.damle] [BEAM-12427] Ignore the AutoValue_* classes generated in "generated"
[zyichi] [BEAM-12437] Fix broken test from missing allow_unsafe_triggers
[noreply] [BEAM-8787] Contribution Guide Improvement (#14477)
------------------------------------------
[...truncated 545.69 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/02 00:12:36 Prepared job with id: go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14 and staging token: go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14
2021/06/02 00:12:36 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622592756040468082
21/06/02 00:12:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.
21/06/02 00:12:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.go.
21/06/02 00:12:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.null.
21/06/02 00:12:39 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.
2021/06/02 00:12:39 Staged binary artifact with token:
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
21/06/02 00:12:39 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
2021/06/02 00:12:39 Submitted job: go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
2021/06/02 00:12:39 Job state: STOPPED
2021/06/02 00:12:39 Job state: STARTING
2021/06/02 00:12:39 Job state: RUNNING
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832 on Spark master local
21/06/02 00:12:39 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832: Pipeline translated successfully. Computing outputs
21/06/02 00:12:39 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46255 ...
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44209 ...
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37357 ...
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 5084419 ns
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 3268731 ns
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e537c06e959c51cdfff023592fc77ffdf0178dd3a9d478962e7243fd6eba55d4/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/02 00:12:41 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container a03996b6b489862fbbd0951f6fd13ea63ca268993b51f8591eedea33e4138eda. Logs:
2021/06/02 00:12:41 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622592756036881929"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622592756036881929-jenkins-0602001239-b9a685d4"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44209"} artifact_endpoint:{url:"localhost:38927"} control_endpoint:{url:"localhost:46255"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e537c06e959c51cdfff023592fc77ffdf0178dd3a9d478962e7243fd6eba55d4/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/02 00:12:41 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40975
2021/06/02 00:12:41 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
2021/06/02 00:12:41 control response channel closed
Remote logging shutting down.Worker exited successfully!
21/06/02 00:12:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832 finished.
2021/06/02 00:12:43 Job state: DONE
21/06/02 00:12:43 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
21/06/02 00:12:43 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
2021/06/02 00:12:43 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (30.79s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 30.817s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.023s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 19s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/bz5w6j3re3x44
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4747
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4747/display/redirect?page=changes>
Changes:
[zyichi] Minor fix to prebuilding sdk workflow timeout setting
[Ismaël Mejía] [BEAM-12423] Upgrade pyarrow to support version 4.0.0 too
------------------------------------------
[...truncated 545.59 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 18:15:05 Prepared job with id: go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8 and staging token: go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8
2021/06/01 18:15:05 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622571305291135607
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.go.
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.null.
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.
2021/06/01 18:15:08 Staged binary artifact with token:
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
21/06/01 18:15:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
2021/06/01 18:15:08 Submitted job: go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
2021/06/01 18:15:08 Job state: STOPPED
2021/06/01 18:15:08 Job state: STARTING
2021/06/01 18:15:08 Job state: RUNNING
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e on Spark master local
21/06/01 18:15:08 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e: Pipeline translated successfully. Computing outputs
21/06/01 18:15:09 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37539 ...
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37473 ...
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37345 ...
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3369206 ns
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1902441 ns
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/eaffad8902bcc2011b67bdfb4f2556d7fe2eddc5af547157766283b182fb692b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 18:15:10 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e837bbee9b0669bf36e667d628cb38df60ced3686c226a90a004c5b3775d7e4b. Logs:
2021/06/01 18:15:10 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622571305288527188"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37473"} artifact_endpoint:{url:"localhost:39421"} control_endpoint:{url:"localhost:37539"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/eaffad8902bcc2011b67bdfb4f2556d7fe2eddc5af547157766283b182fb692b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 18:15:10 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:38387
2021/06/01 18:15:10 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/06/01 18:15:10 control response channel closed
21/06/01 18:15:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e finished.
2021/06/01 18:15:11 Job state: DONE
21/06/01 18:15:11 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
21/06/01 18:15:11 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
2021/06/01 18:15:11 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (34.41s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 34.432s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 45s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/d6qpimrkcdspw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4746
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4746/display/redirect>
Changes:
------------------------------------------
[...truncated 545.77 KB...]
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 12:12:42 Prepared job with id: go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60 and staging token: go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60
2021/06/01 12:12:42 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622549562344721298
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.go.
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.null.
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.
2021/06/01 12:12:44 Staged binary artifact with token:
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
21/06/01 12:12:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
2021/06/01 12:12:44 Submitted job: go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
2021/06/01 12:12:44 Job state: STOPPED
2021/06/01 12:12:44 Job state: STARTING
2021/06/01 12:12:44 Job state: RUNNING
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d on Spark master local
21/06/01 12:12:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d: Pipeline translated successfully. Computing outputs
21/06/01 12:12:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45455 ...
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34893 ...
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37033 ...
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2641799 ns
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1475344 ns
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e5b872fdb855e253be8270f8ceb63d6ff27a375d99ddb8ea467820a268d156a8/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 12:12:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1b0c83e67f62948db6498889c9b2bfd06dfef7c80c5ca00562dba63fc6537254. Logs:
2021/06/01 12:12:46 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622549562342099029"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622549562342099029-jenkins-0601121244-88a2927a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34893"} artifact_endpoint:{url:"localhost:44485"} control_endpoint:{url:"localhost:45455"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e5b872fdb855e253be8270f8ceb63d6ff27a375d99ddb8ea467820a268d156a8/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 12:12:46 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40417
2021/06/01 12:12:46 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/01 12:12:46 control response channel closed
21/06/01 12:12:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d finished.
2021/06/01 12:12:47 Job state: DONE
21/06/01 12:12:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
21/06/01 12:12:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
2021/06/01 12:12:47 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.48s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.494s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 20s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
https://gradle.com/s/h7qjtxg4o3l4q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4745
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4745/display/redirect>
Changes:
------------------------------------------
[...truncated 546.35 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 06:13:17 Prepared job with id: go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208 and staging token: go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208
2021/06/01 06:13:17 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622527997923926760
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.go.
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.null.
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.
2021/06/01 06:13:20 Staged binary artifact with token:
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
21/06/01 06:13:20 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
2021/06/01 06:13:20 Submitted job: go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
2021/06/01 06:13:20 Job state: STOPPED
2021/06/01 06:13:20 Job state: STARTING
2021/06/01 06:13:20 Job state: RUNNING
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1 on Spark master local
21/06/01 06:13:20 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1: Pipeline translated successfully. Computing outputs
21/06/01 06:13:20 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33429 ...
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33361 ...
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44779 ...
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2774368 ns
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1435005 ns
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/63906311f9221f387ca0b03c24aeb74d4c9e63eee4ddb305a5debafe612ca5a3/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 06:13:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 414b39292ab02f1b521b8c4c3d45293b6d78fc01f540e1ade8d048059e95a86b. Logs:
2021/06/01 06:13:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622527997921345339"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622527997921345339-jenkins-0601061320-d7855d32"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33361"} artifact_endpoint:{url:"localhost:35153"} control_endpoint:{url:"localhost:33429"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/63906311f9221f387ca0b03c24aeb74d4c9e63eee4ddb305a5debafe612ca5a3/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 06:13:21 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40595
2021/06/01 06:13:21 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/01 06:13:21 control response channel closed
21/06/01 06:13:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1 finished.
2021/06/01 06:13:22 Job state: DONE
21/06/01 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
21/06/01 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
2021/06/01 06:13:22 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (23.49s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 23.506s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 56s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/mkj4jbn75oxcc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4744
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4744/display/redirect>
Changes:
------------------------------------------
[...truncated 545.89 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 00:13:07 Prepared job with id: go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c and staging token: go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c
2021/06/01 00:13:07 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622506387941605896
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.go.
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.null.
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.
2021/06/01 00:13:10 Staged binary artifact with token:
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
21/06/01 00:13:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
2021/06/01 00:13:10 Submitted job: go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
2021/06/01 00:13:10 Job state: STOPPED
2021/06/01 00:13:10 Job state: STARTING
2021/06/01 00:13:10 Job state: RUNNING
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc on Spark master local
21/06/01 00:13:10 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc: Pipeline translated successfully. Computing outputs
21/06/01 00:13:10 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43579 ...
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45393 ...
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35371 ...
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3036448 ns
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1769681 ns
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/8c93a6c62be388cb97491ee6aa433985d8f1bbc3adfbe56c420feaca49831817/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 00:13:11 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container df085ae8fa951f02f3f03199c793fcdf6fd37d78c7c497321e9cfbc8f2e7d499. Logs:
2021/06/01 00:13:11 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622506387939373424"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622506387939373424-jenkins-0601001310-fbb4149a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45393"} artifact_endpoint:{url:"localhost:44509"} control_endpoint:{url:"localhost:43579"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/8c93a6c62be388cb97491ee6aa433985d8f1bbc3adfbe56c420feaca49831817/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 00:13:11 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:46695
2021/06/01 00:13:11 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/01 00:13:11 control response channel closed
21/06/01 00:13:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc finished.
2021/06/01 00:13:12 Job state: DONE
21/06/01 00:13:12 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
21/06/01 00:13:12 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
2021/06/01 00:13:12 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (23.54s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 23.564s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 47s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/izasu2jlisqai
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4743
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4743/display/redirect>
Changes:
------------------------------------------
[...truncated 545.94 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 18:12:44 Prepared job with id: go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44 and staging token: go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44
2021/05/31 18:12:44 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622484764975225006
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.go.
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.null.
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.
2021/05/31 18:12:47 Staged binary artifact with token:
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
21/05/31 18:12:47 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
2021/05/31 18:12:47 Submitted job: go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
2021/05/31 18:12:47 Job state: STOPPED
2021/05/31 18:12:47 Job state: STARTING
2021/05/31 18:12:47 Job state: RUNNING
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255 on Spark master local
21/05/31 18:12:47 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255: Pipeline translated successfully. Computing outputs
21/05/31 18:12:48 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35419 ...
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46655 ...
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46337 ...
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2933349 ns
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1635793 ns
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/76a835c81f77ba8d4d538a30e678d2727424d1f37c160c1a33885a65c660a02b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 18:12:49 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d3889c99a3722b2e6095a3135eecced6c3251f479bc23e4010794763782667c5. Logs:
2021/05/31 18:12:48 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622484764972370019"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622484764972370019-jenkins-0531181247-6f9350b7"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46655"} artifact_endpoint:{url:"localhost:37755"} control_endpoint:{url:"localhost:35419"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/76a835c81f77ba8d4d538a30e678d2727424d1f37c160c1a33885a65c660a02b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 18:12:48 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44417
2021/05/31 18:12:49 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/31 18:12:49 control response channel closed
21/05/31 18:12:49 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255 finished.
2021/05/31 18:12:49 Job state: DONE
21/05/31 18:12:49 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
21/05/31 18:12:49 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
2021/05/31 18:12:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.47s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.491s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 23s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/ti6spzi4uzars
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4742
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4742/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12277] Update Flink 1.13 version to 1.13.1
[Ismaël Mejía] [BEAM-12424] Update Flink 1.12 to version 1.12.4
------------------------------------------
[...truncated 545.11 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 12:12:12 Prepared job with id: go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a and staging token: go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a
2021/05/31 12:12:12 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622463132233630280
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.go.
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.null.
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.
2021/05/31 12:12:14 Staged binary artifact with token:
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
21/05/31 12:12:14 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
2021/05/31 12:12:14 Submitted job: go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
2021/05/31 12:12:14 Job state: STOPPED
2021/05/31 12:12:14 Job state: STARTING
2021/05/31 12:12:14 Job state: RUNNING
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1 on Spark master local
21/05/31 12:12:14 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1: Pipeline translated successfully. Computing outputs
21/05/31 12:12:15 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45521 ...
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37107 ...
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37695 ...
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2542551 ns
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1420436 ns
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/7f8e7e88a5e9e62c99e16b2a15d24de168a48d59695f56982e342efeddfce32b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 12:12:16 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2ef7d796de5a91fcedc9ad5f14e0111c742c2050f5ddc6a275e6d4d61d5069a8. Logs:
2021/05/31 12:12:16 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622463132231234252"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622463132231234252-jenkins-0531121214-10387377"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37107"} artifact_endpoint:{url:"localhost:43953"} control_endpoint:{url:"localhost:45521"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/7f8e7e88a5e9e62c99e16b2a15d24de168a48d59695f56982e342efeddfce32b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 12:12:16 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44379
2021/05/31 12:12:16 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/31 12:12:16 control response channel closed
21/05/31 12:12:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1 finished.
2021/05/31 12:12:17 Job state: DONE
21/05/31 12:12:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
21/05/31 12:12:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
2021/05/31 12:12:17 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.41s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.426s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 53s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/5kql55mznbpvm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4741
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4741/display/redirect>
Changes:
------------------------------------------
[...truncated 546.67 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 06:13:08 Prepared job with id: go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23 and staging token: go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23
2021/05/31 06:13:08 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622441588542188274
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.go.
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.null.
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.
2021/05/31 06:13:10 Staged binary artifact with token:
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
21/05/31 06:13:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
2021/05/31 06:13:10 Submitted job: go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
2021/05/31 06:13:10 Job state: STOPPED
2021/05/31 06:13:10 Job state: STARTING
2021/05/31 06:13:10 Job state: RUNNING
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0 on Spark master local
21/05/31 06:13:10 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0: Pipeline translated successfully. Computing outputs
21/05/31 06:13:11 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42597 ...
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43983 ...
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37435 ...
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2884294 ns
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1502687 ns
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/bfc4a8156e0981dc9ade400d1fa2c499f57b9636545a7a306392b8640c74dc6b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 06:13:12 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 02566e585ffa101d4275a014b7e2447ee124450ed0983f50a203962a7cc336c2. Logs:
2021/05/31 06:13:12 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622441588539781408"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622441588539781408-jenkins-0531061310-24d36646"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43983"} artifact_endpoint:{url:"localhost:41691"} control_endpoint:{url:"localhost:42597"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/bfc4a8156e0981dc9ade400d1fa2c499f57b9636545a7a306392b8640c74dc6b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 06:13:12 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:42061
2021/05/31 06:13:12 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/31 06:13:12 control response channel closed
21/05/31 06:13:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0 finished.
2021/05/31 06:13:13 Job state: DONE
21/05/31 06:13:13 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
21/05/31 06:13:13 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
2021/05/31 06:13:13 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (23.57s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 23.585s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 45s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/rpdp6zxd7duag
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4740
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4740/display/redirect>
Changes:
------------------------------------------
[...truncated 546.15 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 00:12:59 Prepared job with id: go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd and staging token: go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd
2021/05/31 00:12:59 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622419979457710193
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.go.
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.null.
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.
2021/05/31 00:13:01 Staged binary artifact with token:
21/05/31 00:13:01 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
21/05/31 00:13:01 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
2021/05/31 00:13:01 Submitted job: go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
2021/05/31 00:13:01 Job state: STOPPED
2021/05/31 00:13:01 Job state: STARTING
2021/05/31 00:13:01 Job state: RUNNING
21/05/31 00:13:01 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 00:13:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7 on Spark master local
21/05/31 00:13:02 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 00:13:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7: Pipeline translated successfully. Computing outputs
21/05/31 00:13:02 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40333 ...
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36149 ...
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44273 ...
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2712384 ns
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1521661 ns
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/277eee75898a037c1773cfacebd0ab3f97df4df93c7dc1db3c74806dbc35bfc9/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 00:13:03 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ffcf8e75a0472930a5a986fc0780f19d1f7817e628790fc366fda32219218b50. Logs:
2021/05/31 00:13:03 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622419979455082976"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622419979455082976-jenkins-0531001301-76477bbb"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36149"} artifact_endpoint:{url:"localhost:33525"} control_endpoint:{url:"localhost:40333"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/277eee75898a037c1773cfacebd0ab3f97df4df93c7dc1db3c74806dbc35bfc9/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 00:13:03 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:36863
2021/05/31 00:13:03 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
2021/05/31 00:13:03 control response channel closed
Worker exited successfully!
21/05/31 00:13:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7 finished.
2021/05/31 00:13:04 Job state: DONE
21/05/31 00:13:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
21/05/31 00:13:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
2021/05/31 00:13:04 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.29s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.307s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 39s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/o6iku4jxuupb2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4739
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4739/display/redirect>
Changes:
------------------------------------------
[...truncated 546.10 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 18:13:41 Prepared job with id: go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749 and staging token: go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749
2021/05/30 18:13:41 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622398421600806877
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.go.
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.null.
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.
2021/05/30 18:13:44 Staged binary artifact with token:
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
21/05/30 18:13:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
2021/05/30 18:13:44 Submitted job: go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
2021/05/30 18:13:44 Job state: STOPPED
2021/05/30 18:13:44 Job state: STARTING
2021/05/30 18:13:44 Job state: RUNNING
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3 on Spark master local
21/05/30 18:13:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3: Pipeline translated successfully. Computing outputs
21/05/30 18:13:44 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39137 ...
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43715 ...
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46565 ...
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3094222 ns
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1595540 ns
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/5feb5e47e94346187ea5671698aff984dff21d933cbfd4d81a5f4c2fbbeffc29/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 18:13:45 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2d0a11b7f47a743f3c43c6982512968ad7fd50ff95df35aaae9a49db1610f0f6. Logs:
2021/05/30 18:13:45 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622398421598058738"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622398421598058738-jenkins-0530181344-c9f797d2"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43715"} artifact_endpoint:{url:"localhost:46505"} control_endpoint:{url:"localhost:39137"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/5feb5e47e94346187ea5671698aff984dff21d933cbfd4d81a5f4c2fbbeffc29/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 18:13:45 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:41675
2021/05/30 18:13:45 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 18:13:45 control response channel closed
21/05/30 18:13:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3 finished.
2021/05/30 18:13:46 Job state: DONE
21/05/30 18:13:46 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
21/05/30 18:13:46 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
2021/05/30 18:13:46 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.38s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.402s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 20s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/67hiqbn37wkqa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4738
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4738/display/redirect>
Changes:
------------------------------------------
[...truncated 545.60 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 12:18:38 Prepared job with id: go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7 and staging token: go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7
2021/05/30 12:18:38 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622377118029096917
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.go.
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.null.
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.
21/05/30 12:18:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
21/05/30 12:18:43 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
21/05/30 12:18:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
2021/05/30 12:18:43 Staged binary artifact with token:
2021/05/30 12:18:43 Submitted job: go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
2021/05/30 12:18:43 Job state: STOPPED
2021/05/30 12:18:43 Job state: STARTING
2021/05/30 12:18:43 Job state: RUNNING
21/05/30 12:18:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12 on Spark master local
21/05/30 12:18:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 12:18:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12: Pipeline translated successfully. Computing outputs
21/05/30 12:18:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 12:18:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 12:18:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 12:18:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36521 ...
21/05/30 12:18:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44869 ...
21/05/30 12:18:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 12:18:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44005 ...
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 12:18:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 10570479 ns
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/681d13130eee1d237929b749af12670a818b583834523270f8f417f1052bb9de/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 12:18:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 7593719 ns
21/05/30 12:18:47 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c16b378ca7fc2609daed50fd8f4aa2264288820c7588eb352558816e965fb069. Logs:
2021/05/30 12:18:46 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622377118021005828"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622377118021005828-jenkins-0530121843-2eb720dc"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44869"} artifact_endpoint:{url:"localhost:44871"} control_endpoint:{url:"localhost:36521"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/681d13130eee1d237929b749af12670a818b583834523270f8f417f1052bb9de/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 12:18:46 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:33655
2021/05/30 12:18:46 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 12:18:47 control response channel closed
21/05/30 12:18:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12 finished.
2021/05/30 12:18:48 Job state: DONE
21/05/30 12:18:48 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
21/05/30 12:18:48 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
2021/05/30 12:18:48 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (58.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 58.327s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.059s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 18m 27s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/mfsilndttcdtw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4737
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4737/display/redirect>
Changes:
------------------------------------------
[...truncated 546.12 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 06:13:15 Prepared job with id: go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214 and staging token: go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214
2021/05/30 06:13:15 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622355195971824488
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.go.
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.null.
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.
2021/05/30 06:13:18 Staged binary artifact with token:
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
21/05/30 06:13:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
2021/05/30 06:13:18 Submitted job: go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
2021/05/30 06:13:18 Job state: STOPPED
2021/05/30 06:13:18 Job state: STARTING
2021/05/30 06:13:18 Job state: RUNNING
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98 on Spark master local
21/05/30 06:13:18 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98: Pipeline translated successfully. Computing outputs
21/05/30 06:13:19 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44645 ...
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37447 ...
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38223 ...
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3258757 ns
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1694343 ns
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/2d89b203547beb73d947a2b3290d5c251a09239dce771c5bcfc3940d4eef4a07/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 06:13:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f57ab0038d27c600a5595556044bddd1fbd9fda75914b7451fc41f1be2fd8f5e. Logs:
2021/05/30 06:13:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622355195968983383"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622355195968983383-jenkins-0530061318-18c67c42"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37447"} artifact_endpoint:{url:"localhost:34509"} control_endpoint:{url:"localhost:44645"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/2d89b203547beb73d947a2b3290d5c251a09239dce771c5bcfc3940d4eef4a07/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 06:13:21 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:41087
2021/05/30 06:13:21 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 06:13:21 control response channel closed
21/05/30 06:13:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98 finished.
2021/05/30 06:13:22 Job state: DONE
21/05/30 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
21/05/30 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
2021/05/30 06:13:22 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (30.26s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 30.282s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 42s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/ih4amcevi5xiw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4736
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4736/display/redirect>
Changes:
------------------------------------------
[...truncated 546.02 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 00:13:18 Prepared job with id: go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70 and staging token: go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70
2021/05/30 00:13:18 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622333598502582764
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.go.
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.null.
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.
2021/05/30 00:13:21 Staged binary artifact with token:
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
21/05/30 00:13:21 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
2021/05/30 00:13:21 Submitted job: go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
2021/05/30 00:13:21 Job state: STOPPED
2021/05/30 00:13:21 Job state: STARTING
2021/05/30 00:13:21 Job state: RUNNING
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430 on Spark master local
21/05/30 00:13:21 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430: Pipeline translated successfully. Computing outputs
21/05/30 00:13:21 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46515 ...
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38367 ...
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45473 ...
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2971495 ns
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1645436 ns
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/6ba176465a32f401c71285dccaa6c6dd39b8ee3cc7080f107dbffe2d6e77567a/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 00:13:22 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container afb48784e97f657a52ad32364c5b4aa6205093a3203f865951057f14579985f8. Logs:
2021/05/30 00:13:22 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622333598500075722"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622333598500075722-jenkins-0530001321-bf8d8332"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38367"} artifact_endpoint:{url:"localhost:38393"} control_endpoint:{url:"localhost:46515"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/6ba176465a32f401c71285dccaa6c6dd39b8ee3cc7080f107dbffe2d6e77567a/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 00:13:22 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:37733
2021/05/30 00:13:22 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 00:13:22 control response channel closed
21/05/30 00:13:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430 finished.
2021/05/30 00:13:23 Job state: DONE
21/05/30 00:13:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
21/05/30 00:13:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
2021/05/30 00:13:23 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.87s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.896s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 2s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/vhhbpfpihk5ee
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4735
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4735/display/redirect>
Changes:
------------------------------------------
[...truncated 546.34 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 18:13:40 Prepared job with id: go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd and staging token: go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd
2021/05/29 18:13:40 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622312020339574884
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.go.
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.null.
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.
2021/05/29 18:13:42 Staged binary artifact with token:
21/05/29 18:13:42 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
21/05/29 18:13:42 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
2021/05/29 18:13:43 Submitted job: go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
2021/05/29 18:13:43 Job state: STOPPED
2021/05/29 18:13:43 Job state: STARTING
2021/05/29 18:13:43 Job state: RUNNING
21/05/29 18:13:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/29 18:13:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed on Spark master local
21/05/29 18:13:43 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 18:13:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed: Pipeline translated successfully. Computing outputs
21/05/29 18:13:43 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38779 ...
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35917 ...
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44107 ...
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2966289 ns
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1634259 ns
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e4698c1ccb4f6966ff1f812b3d650c3bcd35acaf438e9cf31b668c6817bdfa39/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 18:13:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 08b7423ed2d2e4661fa3fa9a2f1970ddd02980f30f118d587ce7e397d84ffec4. Logs:
2021/05/29 18:13:44 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622312020336805684"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622312020336805684-jenkins-0529181342-bc8292f8"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35917"} artifact_endpoint:{url:"localhost:38127"} control_endpoint:{url:"localhost:38779"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e4698c1ccb4f6966ff1f812b3d650c3bcd35acaf438e9cf31b668c6817bdfa39/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 18:13:44 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:45931
2021/05/29 18:13:44 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/29 18:13:44 control response channel closed
21/05/29 18:13:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed finished.
2021/05/29 18:13:45 Job state: DONE
21/05/29 18:13:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
21/05/29 18:13:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
2021/05/29 18:13:45 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.45s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.473s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 19s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/vi5mb6asmllaq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4734
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4734/display/redirect>
Changes:
------------------------------------------
[...truncated 545.87 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 12:10:58 Prepared job with id: go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348 and staging token: go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348
2021/05/29 12:10:58 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622290258006291245
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.go.
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.null.
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.
2021/05/29 12:11:00 Staged binary artifact with token:
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
21/05/29 12:11:00 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
2021/05/29 12:11:00 Submitted job: go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
2021/05/29 12:11:00 Job state: STOPPED
2021/05/29 12:11:00 Job state: STARTING
2021/05/29 12:11:00 Job state: RUNNING
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11 on Spark master local
21/05/29 12:11:00 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11: Pipeline translated successfully. Computing outputs
21/05/29 12:11:01 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38499 ...
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44359 ...
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35419 ...
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3291161 ns
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1894607 ns
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/69f8d509ef0235415bac27f52fbe3866b76c883fe650a38dc75109732b5391aa/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 12:11:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 70226da8d782d00608f23c34b12e4588c164abd81ecd5a5f105121fb9c62ccd4. Logs:
2021/05/29 12:11:01 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622290258002805754"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622290258002805754-jenkins-0529121100-1cff0ae2"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44359"} artifact_endpoint:{url:"localhost:46799"} control_endpoint:{url:"localhost:38499"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/69f8d509ef0235415bac27f52fbe3866b76c883fe650a38dc75109732b5391aa/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 12:11:01 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:43855
2021/05/29 12:11:02 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/29 12:11:02 control response channel closed
21/05/29 12:11:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11 finished.
2021/05/29 12:11:03 Job state: DONE
21/05/29 12:11:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
21/05/29 12:11:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
2021/05/29 12:11:03 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (29.16s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 29.183s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 34s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/g7ikxeg5jabbq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4733
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4733/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11608] Development environment set up automation (#14584)
------------------------------------------
[...truncated 546.47 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 06:13:51 Prepared job with id: go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee and staging token: go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee
2021/05/29 06:13:51 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622268831858653583
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.go.
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.null.
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.
2021/05/29 06:13:54 Staged binary artifact with token:
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
21/05/29 06:13:54 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
2021/05/29 06:13:54 Submitted job: go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
2021/05/29 06:13:54 Job state: STOPPED
2021/05/29 06:13:54 Job state: STARTING
2021/05/29 06:13:54 Job state: RUNNING
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083 on Spark master local
21/05/29 06:13:54 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083: Pipeline translated successfully. Computing outputs
21/05/29 06:13:54 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44555 ...
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42063 ...
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40507 ...
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2719116 ns
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1501200 ns
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/f9819db2fd14a32a59cb70a1f08d6efc402dafad294b62e990f2761622b53d87/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 06:13:55 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b8b137c76d97cce61aa2300b66bf9eeda1d76a6d55da395b48b1b96998ec86be. Logs:
2021/05/29 06:13:55 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622268831856323278"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622268831856323278-jenkins-0529061354-601e2e5"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42063"} artifact_endpoint:{url:"localhost:40261"} control_endpoint:{url:"localhost:44555"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/f9819db2fd14a32a59cb70a1f08d6efc402dafad294b62e990f2761622b53d87/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 06:13:55 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44621
2021/05/29 06:13:55 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
2021/05/29 06:13:55 control response channel closed
Worker exited successfully!
21/05/29 06:13:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083 finished.
2021/05/29 06:13:56 Job state: DONE
21/05/29 06:13:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
21/05/29 06:13:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
2021/05/29 06:13:56 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.09s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.111s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 29s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/nleup2fltcxlw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4732
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4732/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11988] added a fix for missing sample.txt (#14889)
[noreply] [BEAM-621] Add MapKeys, MapValues transforms (#14273)
------------------------------------------
[...truncated 547.69 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 00:17:42 Prepared job with id: go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3 and staging token: go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3
2021/05/29 00:17:42 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622247462353935115
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.go.
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.null.
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.
2021/05/29 00:17:48 Staged binary artifact with token:
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
21/05/29 00:17:48 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
2021/05/29 00:17:48 Submitted job: go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
2021/05/29 00:17:48 Job state: STOPPED
2021/05/29 00:17:48 Job state: STARTING
2021/05/29 00:17:48 Job state: RUNNING
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1 on Spark master local
21/05/29 00:17:48 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1: Pipeline translated successfully. Computing outputs
21/05/29 00:17:49 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33633 ...
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37227 ...
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40735 ...
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 4823301 ns
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2646354 ns
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/1593fbe8e99ce1cb89a692e92d95acb14f6f105d25ca5bfa62774ba15a1bcbde/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 00:17:51 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container eaa36c661592ea9c019f5d285bcad1285ffde606de24b7cee4c827b814bc4f1d. Logs:
2021/05/29 00:17:50 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622247462350129770"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622247462350129770-jenkins-0529001748-b15eb165"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37227"} artifact_endpoint:{url:"localhost:33175"} control_endpoint:{url:"localhost:33633"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/1593fbe8e99ce1cb89a692e92d95acb14f6f105d25ca5bfa62774ba15a1bcbde/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 00:17:50 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:43787
2021/05/29 00:17:50 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/29 00:17:51 control response channel closed
21/05/29 00:17:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1 finished.
2021/05/29 00:17:53 Job state: DONE
21/05/29 00:17:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
21/05/29 00:17:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
2021/05/29 00:17:53 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (41.97s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 41.996s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 17m 29s
79 actionable tasks: 62 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/hliyefdkupi2w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4731
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4731/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12277] Add flink 1.13 build target.
[Ismaël Mejía] [BEAM-12277] Make Flink 1.13 the default Flink version to run the tests
[Ismaël Mejía] [BEAM-12277] Update expected metric name formatting.
[Ismaël Mejía] [BEAM-12277] fix mistake in source overrides
------------------------------------------
[...truncated 547.22 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 18:16:51 Prepared job with id: go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753 and staging token: go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753
2021/05/28 18:16:51 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622225811470283233
21/05/28 18:16:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.
21/05/28 18:16:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.go.
21/05/28 18:16:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.null.
21/05/28 18:16:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.
2021/05/28 18:16:54 Staged binary artifact with token:
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
21/05/28 18:16:54 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
2021/05/28 18:16:54 Submitted job: go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
2021/05/28 18:16:54 Job state: STOPPED
2021/05/28 18:16:54 Job state: STARTING
2021/05/28 18:16:54 Job state: RUNNING
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb on Spark master local
21/05/28 18:16:54 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb: Pipeline translated successfully. Computing outputs
21/05/28 18:16:54 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37135 ...
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37455 ...
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35795 ...
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3006785 ns
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1593739 ns
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/98fd382c021abc3ae4904aa88b522d4143ea96511afec468a0dfa42ca7d26855/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 18:16:55 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 20db6c0acab9e291538d4dc45c057d109ce8f158b505a92aca468a3153b961d6. Logs:
2021/05/28 18:16:55 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622225811467135324"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622225811467135324-jenkins-0528181654-e74bff0d"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37455"} artifact_endpoint:{url:"localhost:35115"} control_endpoint:{url:"localhost:37135"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/98fd382c021abc3ae4904aa88b522d4143ea96511afec468a0dfa42ca7d26855/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 18:16:55 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:38225
2021/05/28 18:16:55 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/28 18:16:55 control response channel closed
21/05/28 18:16:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb finished.
2021/05/28 18:16:56 Job state: DONE
21/05/28 18:16:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
21/05/28 18:16:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
2021/05/28 18:16:56 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.30s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.324s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 28s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/szajnpq3vmsa4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4730
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4730/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12384] Refine generic types on Read.Bounded internals
[Ismaël Mejía] [BEAM-12384] Set output typeDescriptor explictly in Read.Bounded
[Ismaël Mejía] [BEAM-12384] Infer typeDescriptor from coder if typeDescriptor is not
[Ismaël Mejía] [BEAM-12415] Update Spark 3 version to 3.1.2
------------------------------------------
[...truncated 548.35 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 12:22:12 Prepared job with id: go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521 and staging token: go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521
2021/05/28 12:22:12 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622204532388027149
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.go.
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.null.
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.
2021/05/28 12:22:15 Staged binary artifact with token:
21/05/28 12:22:15 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
21/05/28 12:22:15 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
2021/05/28 12:22:15 Submitted job: go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
2021/05/28 12:22:15 Job state: STOPPED
2021/05/28 12:22:15 Job state: STARTING
2021/05/28 12:22:15 Job state: RUNNING
21/05/28 12:22:15 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 12:22:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48 on Spark master local
21/05/28 12:22:16 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 12:22:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48: Pipeline translated successfully. Computing outputs
21/05/28 12:22:16 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46197 ...
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40195 ...
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40595 ...
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3179692 ns
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1642072 ns
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/84d2ae7678f1781c628b9a838b95fd94e72e698a7e8e8012ec50764e1cfe32bb/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 12:22:18 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 5f212d1ccc1582ded45240c3894931584b9fac14642746385d8632cb64cd3fba. Logs:
2021/05/28 12:22:17 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622204532385149167"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622204532385149167-jenkins-0528122215-6d46ca07"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40195"} artifact_endpoint:{url:"localhost:45643"} control_endpoint:{url:"localhost:46197"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/84d2ae7678f1781c628b9a838b95fd94e72e698a7e8e8012ec50764e1cfe32bb/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 12:22:17 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40707
2021/05/28 12:22:18 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/28 12:22:18 control response channel closed
21/05/28 12:22:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48 finished.
2021/05/28 12:22:19 Job state: DONE
21/05/28 12:22:19 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
21/05/28 12:22:19 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
2021/05/28 12:22:19 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (33.23s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 33.254s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 21m 55s
79 actionable tasks: 69 executed, 10 from cache
Publishing build scan...
https://gradle.com/s/myizweq6jf72w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4729
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4729/display/redirect?page=changes>
Changes:
[Boyuan Zhang] Populate use_sdf_read in ExpansionService.
------------------------------------------
[...truncated 546.66 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 06:16:15 Prepared job with id: go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4 and staging token: go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4
2021/05/28 06:16:15 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622182575257977391
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.go.
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.null.
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.
2021/05/28 06:16:17 Staged binary artifact with token:
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
21/05/28 06:16:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
2021/05/28 06:16:18 Submitted job: go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
2021/05/28 06:16:18 Job state: STOPPED
2021/05/28 06:16:18 Job state: STARTING
2021/05/28 06:16:18 Job state: RUNNING
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f on Spark master local
21/05/28 06:16:18 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f: Pipeline translated successfully. Computing outputs
21/05/28 06:16:18 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43305 ...
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34709 ...
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42331 ...
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3325343 ns
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1795198 ns
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/ea46141e183d214dd0c476f3efcaf464a89cceb21fa30390daece879e82d2624/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 06:16:19 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e21ebcb6bd63d9c786feb5ded9457b71869e28d3655b28c65556a899f8578be9. Logs:
2021/05/28 06:16:19 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622182575254318625"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622182575254318625-jenkins-0528061618-870d093"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34709"} artifact_endpoint:{url:"localhost:33397"} control_endpoint:{url:"localhost:43305"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/ea46141e183d214dd0c476f3efcaf464a89cceb21fa30390daece879e82d2624/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 06:16:19 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:37321
2021/05/28 06:16:19 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/28 06:16:19 control response channel closed
21/05/28 06:16:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f finished.
2021/05/28 06:16:20 Job state: DONE
21/05/28 06:16:20 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
21/05/28 06:16:20 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
2021/05/28 06:16:20 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.96s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.977s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 15m 55s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/n2x5fizbjn2iq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4728
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4728/display/redirect?page=changes>
Changes:
[zhoufek] [BEAM-9487] Add trigger safety check to GroupByKey
[noreply] [BEAM-12370] Support side input in Samza portable runner (#14883)
[noreply] [BEAM-12158][BEAM-12341] Pubsub fixes. (#14896)
------------------------------------------
[...truncated 548.04 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 00:17:13 Prepared job with id: go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c and staging token: go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c
2021/05/28 00:17:13 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622161033648780657
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.go.
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.null.
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.
2021/05/28 00:17:18 Staged binary artifact with token:
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
21/05/28 00:17:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
2021/05/28 00:17:18 Submitted job: go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
2021/05/28 00:17:18 Job state: STOPPED
2021/05/28 00:17:18 Job state: STARTING
2021/05/28 00:17:18 Job state: RUNNING
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22 on Spark master local
21/05/28 00:17:18 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22: Pipeline translated successfully. Computing outputs
21/05/28 00:17:19 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45963 ...
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:41357 ...
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43751 ...
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3618762 ns
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2029276 ns
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/774f85400719e8b76c5eb8eeda0e183042ae29f0ff07ca1434cf6f64cbdbdd9d/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 00:17:20 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6e40c636e7a3c7c510deb8741d4f788400f17e4d9e3845002b20ac02bf30c2d8. Logs:
2021/05/28 00:17:20 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622161033644978888"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622161033644978888-jenkins-0528001718-767617c4"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41357"} artifact_endpoint:{url:"localhost:40391"} control_endpoint:{url:"localhost:45963"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/774f85400719e8b76c5eb8eeda0e183042ae29f0ff07ca1434cf6f64cbdbdd9d/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 00:17:20 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:45179
2021/05/28 00:17:20 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/28 00:17:20 control response channel closed
21/05/28 00:17:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22 finished.
2021/05/28 00:17:21 Job state: DONE
21/05/28 00:17:21 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
21/05/28 00:17:21 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
2021/05/28 00:17:21 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (41.20s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 41.228s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.022s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 57s
79 actionable tasks: 58 executed, 21 from cache
Publishing build scan...
https://gradle.com/s/zfvgk3h62pf6m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4727
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4727/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12093] Overhaul ElasticsearchIO.Write (#14347)
------------------------------------------
[...truncated 564.52 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 18:14:51 Prepared job with id: go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d and staging token: go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d
2021/05/27 18:14:51 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622139291208428050
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.go.
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.null.
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.
2021/05/27 18:14:53 Staged binary artifact with token:
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
21/05/27 18:14:53 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
2021/05/27 18:14:53 Submitted job: go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
2021/05/27 18:14:53 Job state: STOPPED
2021/05/27 18:14:53 Job state: STARTING
2021/05/27 18:14:53 Job state: RUNNING
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef on Spark master local
21/05/27 18:14:53 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef: Pipeline translated successfully. Computing outputs
21/05/27 18:14:54 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:41711 ...
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39011 ...
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40461 ...
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2999768 ns
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1562033 ns
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/f7e6ff6ade42008b46ebf8efe2f87707c41556a5534c44deaf09c620ae930aca/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 18:14:57 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6c2c4adefc49e2a365ebc407b3cb92cea8489366a8e13e13bac261ad28406915. Logs:
2021/05/27 18:14:57 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622139291205489057"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622139291205489057-jenkins-0527181453-e43af5b2"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39011"} artifact_endpoint:{url:"localhost:40735"} control_endpoint:{url:"localhost:41711"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/f7e6ff6ade42008b46ebf8efe2f87707c41556a5534c44deaf09c620ae930aca/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 18:14:57 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34465
2021/05/27 18:14:57 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/27 18:14:57 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 18:14:57 control response channel closed
21/05/27 18:15:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef finished.
2021/05/27 18:15:02 Job state: DONE
21/05/27 18:15:02 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
21/05/27 18:15:02 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
2021/05/27 18:15:02 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (38.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 38.263s
2021/05/27 18:15:03 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.020s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 39s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/vqrzs67hrg2bq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4726
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4726/display/redirect?page=changes>
Changes:
[je.ik] [BEAM-12276] Narrow check for output timestamp of event time timers
------------------------------------------
[...truncated 564.48 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 12:11:24 Prepared job with id: go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91 and staging token: go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91
2021/05/27 12:11:24 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622117484070716378
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.go.
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.null.
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.
2021/05/27 12:11:26 Staged binary artifact with token:
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
21/05/27 12:11:26 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
2021/05/27 12:11:26 Submitted job: go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
2021/05/27 12:11:26 Job state: STOPPED
2021/05/27 12:11:26 Job state: STARTING
2021/05/27 12:11:26 Job state: RUNNING
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45 on Spark master local
21/05/27 12:11:26 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45: Pipeline translated successfully. Computing outputs
21/05/27 12:11:27 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45393 ...
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45103 ...
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39611 ...
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2901406 ns
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1665638 ns
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/8470380370064b97241b917bcd4bada56bd64f833352bd76a96cf8a7064fceee/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 12:11:29 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 588290e39b9feba425623539839a06f3488d67ef3d664af4f4262706a1ccb668. Logs:
2021/05/27 12:11:29 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622117484068346854"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622117484068346854-jenkins-0527121126-6489ae1e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45103"} artifact_endpoint:{url:"localhost:45895"} control_endpoint:{url:"localhost:45393"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/8470380370064b97241b917bcd4bada56bd64f833352bd76a96cf8a7064fceee/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 12:11:29 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44243
2021/05/27 12:11:29 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/27 12:11:29 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 12:11:29 control response channel closed
21/05/27 12:11:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45 finished.
2021/05/27 12:11:30 Job state: DONE
21/05/27 12:11:30 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
21/05/27 12:11:30 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
2021/05/27 12:11:30 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (30.21s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 30.234s
2021/05/27 12:11:32 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 9s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/3ymbsip2iuqds
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4725
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4725/display/redirect>
Changes:
------------------------------------------
[...truncated 565.04 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 06:14:40 Prepared job with id: go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea and staging token: go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea
2021/05/27 06:14:40 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622096080096767958
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.go.
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.null.
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.
2021/05/27 06:14:43 Staged binary artifact with token:
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
21/05/27 06:14:43 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
2021/05/27 06:14:43 Submitted job: go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
2021/05/27 06:14:43 Job state: STOPPED
2021/05/27 06:14:43 Job state: STARTING
2021/05/27 06:14:43 Job state: RUNNING
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae on Spark master local
21/05/27 06:14:43 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae: Pipeline translated successfully. Computing outputs
21/05/27 06:14:43 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37627 ...
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34229 ...
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42983 ...
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3522117 ns
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1902962 ns
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/7fb59c4b154073c19eabd6391454962ce7c2ac446748af7f7a4b6486360181b8/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 06:14:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ce25e6dc2628cb4966dbe8a631439f6a83401d47f38b7941a59a3c7a064ff479. Logs:
2021/05/27 06:14:44 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622096080093745755"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622096080093745755-jenkins-0527061443-f2adbfe6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34229"} artifact_endpoint:{url:"localhost:35213"} control_endpoint:{url:"localhost:37627"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/7fb59c4b154073c19eabd6391454962ce7c2ac446748af7f7a4b6486360181b8/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 06:14:44 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:35215
2021/05/27 06:14:44 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/27 06:14:44 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 06:14:44 control response channel closed
21/05/27 06:14:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae finished.
2021/05/27 06:14:45 Job state: DONE
21/05/27 06:14:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
21/05/27 06:14:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
2021/05/27 06:14:45 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.57s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.595s
2021/05/27 06:14:47 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.021s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 19s
79 actionable tasks: 53 executed, 26 from cache
Publishing build scan...
https://gradle.com/s/qobi2yab7w3lg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4724
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4724/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12240] Add Java 17 as a valid environment
[emilyye] update release tag for Java UW container to not have beam-prefix
[zyichi] Bump dataflow java legacy container version to beam-master-20210525
[zyichi] Bump python container versions to beam-master-20210526
[noreply] [BEAM-9547] Add additional unliftable grouped aggregations, add describe
[noreply] [BEAM-12024] Add DataFrame example pipeline README (#14886)
[noreply] Merge pull request #14829 from [BEAM-11838] An x-lang wrapper for
[noreply] [BEAM-12074] Define docstrings for all Series and StringMethods
------------------------------------------
[...truncated 566.29 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 00:17:02 Prepared job with id: go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1 and staging token: go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1
2021/05/27 00:17:02 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622074622729723323
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.go.
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.null.
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.
2021/05/27 00:17:06 Staged binary artifact with token:
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
21/05/27 00:17:06 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
2021/05/27 00:17:06 Submitted job: go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
2021/05/27 00:17:06 Job state: STOPPED
2021/05/27 00:17:06 Job state: STARTING
2021/05/27 00:17:06 Job state: RUNNING
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb on Spark master local
21/05/27 00:17:06 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb: Pipeline translated successfully. Computing outputs
21/05/27 00:17:06 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39303 ...
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46153 ...
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46549 ...
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3247548 ns
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1811480 ns
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e7d213f052690ea783fd8910492d636694fcfb1811f27e7d655fbc7291dee603/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 00:17:16 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6280802e47bbf4152ef4672fd9a71aa82ce89dd753df63a2cf502236a1f98e0b. Logs:
2021/05/27 00:17:16 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622074622726624809"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622074622726624809-jenkins-0527001706-1710cb5e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46153"} artifact_endpoint:{url:"localhost:44745"} control_endpoint:{url:"localhost:39303"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e7d213f052690ea783fd8910492d636694fcfb1811f27e7d655fbc7291dee603/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 00:17:16 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:43623
2021/05/27 00:17:16 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/27 00:17:16 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 00:17:16 control response channel closed
21/05/27 00:17:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb finished.
2021/05/27 00:17:17 Job state: DONE
21/05/27 00:17:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
21/05/27 00:17:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
2021/05/27 00:17:17 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (40.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 40.260s
2021/05/27 00:17:19 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 51s
79 actionable tasks: 62 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/xblath2yd73fg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4723
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4723/display/redirect>
Changes:
------------------------------------------
[...truncated 564.52 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 18:14:09 Prepared job with id: go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae and staging token: go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae
2021/05/26 18:14:09 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622052849798635224
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.go.
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.null.
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.
2021/05/26 18:14:12 Staged binary artifact with token:
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
21/05/26 18:14:12 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
2021/05/26 18:14:12 Submitted job: go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
2021/05/26 18:14:12 Job state: STOPPED
2021/05/26 18:14:12 Job state: STARTING
2021/05/26 18:14:12 Job state: RUNNING
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f on Spark master local
21/05/26 18:14:12 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f: Pipeline translated successfully. Computing outputs
21/05/26 18:14:13 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39913 ...
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33611 ...
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33919 ...
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3500968 ns
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1817399 ns
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e885e975aaad63eadee37971f87ef45a0ed1daa9929407170bb622cfa446b93d/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 18:14:14 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container caeef51a5ab9e13aa68daf181d4d1b9d671466df32cdb5b25b33a9a15e6ee23d. Logs:
2021/05/26 18:14:14 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622052849795815379"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622052849795815379-jenkins-0526181412-1d9edd4a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33611"} artifact_endpoint:{url:"localhost:41671"} control_endpoint:{url:"localhost:39913"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e885e975aaad63eadee37971f87ef45a0ed1daa9929407170bb622cfa446b93d/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 18:14:14 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40505
2021/05/26 18:14:14 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 18:14:14 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 18:14:14 control response channel closed
21/05/26 18:14:15 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f finished.
2021/05/26 18:14:15 Job state: DONE
21/05/26 18:14:15 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
21/05/26 18:14:15 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
2021/05/26 18:14:15 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.55s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.576s
2021/05/26 18:14:16 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.020s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 48s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/e4n4wfoh5l2pu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4722
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4722/display/redirect>
Changes:
------------------------------------------
[...truncated 564.58 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 12:10:49 Prepared job with id: go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c and staging token: go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c
2021/05/26 12:10:49 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622031049688025451
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.go.
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.null.
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.
2021/05/26 12:10:52 Staged binary artifact with token:
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
21/05/26 12:10:52 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
2021/05/26 12:10:52 Submitted job: go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
2021/05/26 12:10:52 Job state: STOPPED
2021/05/26 12:10:52 Job state: STARTING
2021/05/26 12:10:52 Job state: RUNNING
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8 on Spark master local
21/05/26 12:10:52 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8: Pipeline translated successfully. Computing outputs
21/05/26 12:10:53 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:32883 ...
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42805 ...
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36755 ...
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3319676 ns
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1762913 ns
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e79d94334f184a5b1f4e0db8be3a774d535c539243140f0ddeb0b83c2e69c683/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 12:10:54 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e1127a5cfaae0a7c9ec06cb6b1feee9d678f73758314afbf026f0b883cba706b. Logs:
2021/05/26 12:10:53 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622031049684592910"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622031049684592910-jenkins-0526121052-22fb2554"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42805"} artifact_endpoint:{url:"localhost:41073"} control_endpoint:{url:"localhost:32883"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e79d94334f184a5b1f4e0db8be3a774d535c539243140f0ddeb0b83c2e69c683/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 12:10:53 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:33913
2021/05/26 12:10:54 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 12:10:54 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 12:10:54 control response channel closed
21/05/26 12:10:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8 finished.
2021/05/26 12:10:54 Job state: DONE
21/05/26 12:10:54 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
21/05/26 12:10:54 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
2021/05/26 12:10:54 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.258s
2021/05/26 12:10:56 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 28s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/tvwfpztqwgtea
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4721
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4721/display/redirect?page=changes>
Changes:
[yoshiki.obata] [BEAM-7372] cleanup py2 codepath
[noreply] [BEAM-12107] Add integration test script for taxiride example (#14882)
[noreply] Add dataflow, tfx to powered by page (#14820)
[noreply] Merge pull request #14765: [BEAM-12405] Reduce tail latency in Storage
------------------------------------------
[...truncated 566.43 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 06:15:59 Prepared job with id: go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8 and staging token: go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8
2021/05/26 06:15:59 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622009759142978740
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.go.
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.null.
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.
2021/05/26 06:16:02 Staged binary artifact with token:
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
21/05/26 06:16:02 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
2021/05/26 06:16:02 Submitted job: go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
2021/05/26 06:16:02 Job state: STOPPED
2021/05/26 06:16:02 Job state: STARTING
2021/05/26 06:16:02 Job state: RUNNING
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce on Spark master local
21/05/26 06:16:02 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce: Pipeline translated successfully. Computing outputs
21/05/26 06:16:02 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34273 ...
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40191 ...
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:32793 ...
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3414942 ns
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1857321 ns
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/82e636dc39d28d125f85714f473df91197f34f800d8a34de63d97c19659f82f0/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 06:16:04 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container eff2e956f5d1304993a7db527cdef921194359561bd1b3cad78c1436c7ebf5a6. Logs:
2021/05/26 06:16:03 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622009759139804762"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622009759139804762-jenkins-0526061602-2eba000"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40191"} artifact_endpoint:{url:"localhost:37403"} control_endpoint:{url:"localhost:34273"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/82e636dc39d28d125f85714f473df91197f34f800d8a34de63d97c19659f82f0/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 06:16:03 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34919
2021/05/26 06:16:03 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 06:16:04 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 06:16:04 control response channel closed
21/05/26 06:16:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce finished.
2021/05/26 06:16:05 Job state: DONE
21/05/26 06:16:05 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
21/05/26 06:16:05 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
2021/05/26 06:16:05 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (32.62s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 32.641s
2021/05/26 06:16:06 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.022s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 15m 37s
79 actionable tasks: 56 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/iuemtv23ugtje
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4720
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4720/display/redirect?page=changes>
Changes:
[relax] GroupIntoBatches improvements
[relax] make sure that DataflowRunner batch override for GroupIntoBatches
[noreply] Minor: Fix doc left nav on dev website (#14881)
[zyichi] [BEAM-12395] Add support to enable google cloud profiler for python SDK
[relax] fix DataflowRunnerTest
[noreply] Fix flaky test on slow remote systems by adding `afterIterations` to
------------------------------------------
[...truncated 567.43 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 00:22:52 Prepared job with id: go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f and staging token: go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f
2021/05/26 00:22:52 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621988572537355649
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.go.
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.null.
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.
2021/05/26 00:22:59 Staged binary artifact with token:
21/05/26 00:22:59 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
21/05/26 00:22:59 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
2021/05/26 00:22:59 Submitted job: go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
2021/05/26 00:22:59 Job state: STOPPED
2021/05/26 00:22:59 Job state: STARTING
2021/05/26 00:22:59 Job state: RUNNING
21/05/26 00:22:59 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 00:23:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb on Spark master local
21/05/26 00:23:00 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 00:23:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb: Pipeline translated successfully. Computing outputs
21/05/26 00:23:01 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34863 ...
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42685 ...
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33965 ...
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 6534702 ns
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 4091085 ns
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/c52d50c5432d2f88f1f1fc9896bc5215d5abd233c62f9cbde81036057f9ebae1/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 00:23:03 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 76fca25bca8b75423e0faf30b1999f18b965421f90bea91c065a752a0a39bb2c. Logs:
2021/05/26 00:23:02 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621988572532376737"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621988572532376737-jenkins-0526002259-75c7f446"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42685"} artifact_endpoint:{url:"localhost:33745"} control_endpoint:{url:"localhost:34863"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/c52d50c5432d2f88f1f1fc9896bc5215d5abd233c62f9cbde81036057f9ebae1/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 00:23:02 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34815
2021/05/26 00:23:03 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 00:23:03 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 00:23:03 control response channel closed
21/05/26 00:23:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb finished.
2021/05/26 00:23:04 Job state: DONE
21/05/26 00:23:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
21/05/26 00:23:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
2021/05/26 00:23:04 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (60.30s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 60.361s
2021/05/26 00:23:09 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.074s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 22m 47s
79 actionable tasks: 69 executed, 10 from cache
Publishing build scan...
https://gradle.com/s/ddfpri3clq7b6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org