You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/05/22 00:16:27 UTC
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4704
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4704/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12339] Implement CREATE FUNCTION statement in Calcite dialect.
[Kyle Weaver] [BEAM-12339] Move unparse tests to different class to avoid 'pipeline
[Kyle Weaver] Add Python Dataflow V2 test to jobs list.
[noreply] [BEAM-12246] Fix ib.collect(dataframe) indexing (#14778)
------------------------------------------
[...truncated 564.71 KB...]
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/22 00:16:12 Prepared job with id: go-job-5-1621642572316398697_05e336f2-37a9-462b-8bea-184d768e1cb4 and staging token: go-job-5-1621642572316398697_05e336f2-37a9-462b-8bea-184d768e1cb4
2021/05/22 00:16:12 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621642572321450729
21/05/22 00:16:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621642572316398697_05e336f2-37a9-462b-8bea-184d768e1cb4.
21/05/22 00:16:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621642572316398697_05e336f2-37a9-462b-8bea-184d768e1cb4.go.
21/05/22 00:16:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621642572316398697_05e336f2-37a9-462b-8bea-184d768e1cb4.null.
21/05/22 00:16:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621642572316398697_05e336f2-37a9-462b-8bea-184d768e1cb4.
2021/05/22 00:16:16 Staged binary artifact with token:
21/05/22 00:16:16 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f
21/05/22 00:16:16 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f
2021/05/22 00:16:16 Submitted job: go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f
2021/05/22 00:16:16 Job state: STOPPED
2021/05/22 00:16:16 Job state: STARTING
2021/05/22 00:16:16 Job state: RUNNING
21/05/22 00:16:16 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/22 00:16:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f on Spark master local
21/05/22 00:16:17 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/22 00:16:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f: Pipeline translated successfully. Computing outputs
21/05/22 00:16:17 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/22 00:16:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/22 00:16:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/22 00:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39855 ...
21/05/22 00:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46409 ...
21/05/22 00:16:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/22 00:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42251 ...
21/05/22 00:16:19 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/22 00:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 4865899 ns
21/05/22 00:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2781063 ns
21/05/22 00:16:19 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/217dfedf3e31d2e4c2aa7fafe34f62f27661c32ccabe6d90f57fe38b09b39b54/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/22 00:16:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/22 00:16:19 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/22 00:16:19 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b1d876ef8bad64532a7aff356796533eae28da4a38d466909c311ee586ecbe4a. Logs:
2021/05/22 00:16:18 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621642572316398697"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621642572316398697-jenkins-0522001616-569b491b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46409"} artifact_endpoint:{url:"localhost:41593"} control_endpoint:{url:"localhost:39855"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/217dfedf3e31d2e4c2aa7fafe34f62f27661c32ccabe6d90f57fe38b09b39b54/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/22 00:16:18 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:45605
2021/05/22 00:16:19 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/22 00:16:19 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/22 00:16:19 control response channel closed
21/05/22 00:16:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f finished.
2021/05/22 00:16:20 Job state: DONE
21/05/22 00:16:20 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f
21/05/22 00:16:20 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621642572316398697-jenkins-0522001616-569b491b_b8c21f47-5860-4028-9643-5b025f65777f
2021/05/22 00:16:20 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (43.94s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 43.988s
2021/05/22 00:16:23 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.054s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 3s
79 actionable tasks: 52 executed, 27 from cache
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/xhu5fqxyzeyzo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Go_VR_Spark #4751
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4751/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4750
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4750/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12281] Drop support for Flink 1.10
[Ismaël Mejía] [BEAM-12281] Update Flink Jenkins jobs to use Flink 1.12
------------------------------------------
[...truncated 545.66 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/02 12:11:05 Prepared job with id: go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813 and staging token: go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813
2021/06/02 12:11:05 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622635865887115713
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.go.
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.null.
21/06/02 12:11:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622635865884473617_bf7a6458-0d33-4586-9af3-ee2efac98813.
2021/06/02 12:11:08 Staged binary artifact with token:
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
21/06/02 12:11:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
2021/06/02 12:11:08 Submitted job: go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
2021/06/02 12:11:08 Job state: STOPPED
2021/06/02 12:11:08 Job state: STARTING
2021/06/02 12:11:08 Job state: RUNNING
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7 on Spark master local
21/06/02 12:11:08 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/02 12:11:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7: Pipeline translated successfully. Computing outputs
21/06/02 12:11:09 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35417 ...
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44973 ...
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36153 ...
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2828080 ns
21/06/02 12:11:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1552704 ns
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e0172f1adb46b13234df41f900e309ca543aa0fc456b98534e1a9d7a46240b88/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/02 12:11:10 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/02 12:11:10 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 8ecfabecf337a1a2b40b86aabed06600cea071b40368848d46e739ee2f14ab61. Logs:
2021/06/02 12:11:09 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622635865884473617"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622635865884473617-jenkins-0602121108-b9829dd5"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44973"} artifact_endpoint:{url:"localhost:34739"} control_endpoint:{url:"localhost:35417"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e0172f1adb46b13234df41f900e309ca543aa0fc456b98534e1a9d7a46240b88/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/02 12:11:09 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:36921
2021/06/02 12:11:10 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/02 12:11:10 control response channel closed
21/06/02 12:11:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7 finished.
2021/06/02 12:11:10 Job state: DONE
21/06/02 12:11:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
21/06/02 12:11:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622635865884473617-jenkins-0602121108-b9829dd5_b872aef9-912c-4c1f-8f10-17b4e58475b7
2021/06/02 12:11:10 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (45.42s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 45.438s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.020s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 45s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/veaqtn3rt5muo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4749
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4749/display/redirect?page=changes>
Changes:
[Brian Hulette] Add DataFrame API updates to CHANGES.md
------------------------------------------
[...truncated 545.97 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/02 06:12:58 Prepared job with id: go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617 and staging token: go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617
2021/06/02 06:12:58 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622614378322587806
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.go.
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.null.
21/06/02 06:13:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622614378320056534_b1eb2985-ca77-430f-9d5d-e5ec34979617.
2021/06/02 06:13:00 Staged binary artifact with token:
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
21/06/02 06:13:00 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
2021/06/02 06:13:00 Submitted job: go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
2021/06/02 06:13:00 Job state: STOPPED
2021/06/02 06:13:00 Job state: STARTING
2021/06/02 06:13:00 Job state: RUNNING
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9 on Spark master local
21/06/02 06:13:00 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/02 06:13:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9: Pipeline translated successfully. Computing outputs
21/06/02 06:13:01 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45403 ...
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:41489 ...
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39363 ...
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2433466 ns
21/06/02 06:13:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1340047 ns
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/584d750791270944e481cc9948ee9d414e4d4bf9f5e44c170587553430ecc14b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/02 06:13:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/02 06:13:02 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container bbf5ffd687f32aaef8fa2f9407b92b0f09a1a3d4e31c46c347386649c514c5d5. Logs:
2021/06/02 06:13:02 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622614378320056534"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622614378320056534-jenkins-0602061300-152bfb7a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41489"} artifact_endpoint:{url:"localhost:43555"} control_endpoint:{url:"localhost:45403"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/584d750791270944e481cc9948ee9d414e4d4bf9f5e44c170587553430ecc14b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/02 06:13:02 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:42195
2021/06/02 06:13:02 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
2021/06/02 06:13:02 control response channel closed
Worker exited successfully!
21/06/02 06:13:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9 finished.
2021/06/02 06:13:03 Job state: DONE
21/06/02 06:13:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
21/06/02 06:13:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622614378320056534-jenkins-0602061300-152bfb7a_e23bc67f-3b38-483c-949f-dca4c369cab9
2021/06/02 06:13:03 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.37s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.397s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 36s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/swok6bh6cfj7m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4748
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4748/display/redirect?page=changes>
Changes:
[anant.damle] [BEAM-12427] Ignore the AutoValue_* classes generated in "generated"
[zyichi] [BEAM-12437] Fix broken test from missing allow_unsafe_triggers
[noreply] [BEAM-8787] Contribution Guide Improvement (#14477)
------------------------------------------
[...truncated 545.69 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/02 00:12:36 Prepared job with id: go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14 and staging token: go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14
2021/06/02 00:12:36 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622592756040468082
21/06/02 00:12:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.
21/06/02 00:12:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.go.
21/06/02 00:12:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.null.
21/06/02 00:12:39 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622592756036881929_9b4a4f1e-bb87-4ad8-a3de-98074dabbe14.
2021/06/02 00:12:39 Staged binary artifact with token:
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
21/06/02 00:12:39 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
2021/06/02 00:12:39 Submitted job: go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
2021/06/02 00:12:39 Job state: STOPPED
2021/06/02 00:12:39 Job state: STARTING
2021/06/02 00:12:39 Job state: RUNNING
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832 on Spark master local
21/06/02 00:12:39 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/02 00:12:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832: Pipeline translated successfully. Computing outputs
21/06/02 00:12:39 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46255 ...
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44209 ...
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37357 ...
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 5084419 ns
21/06/02 00:12:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 3268731 ns
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e537c06e959c51cdfff023592fc77ffdf0178dd3a9d478962e7243fd6eba55d4/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/02 00:12:41 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/02 00:12:41 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container a03996b6b489862fbbd0951f6fd13ea63ca268993b51f8591eedea33e4138eda. Logs:
2021/06/02 00:12:41 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622592756036881929"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622592756036881929-jenkins-0602001239-b9a685d4"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44209"} artifact_endpoint:{url:"localhost:38927"} control_endpoint:{url:"localhost:46255"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e537c06e959c51cdfff023592fc77ffdf0178dd3a9d478962e7243fd6eba55d4/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/02 00:12:41 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40975
2021/06/02 00:12:41 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
2021/06/02 00:12:41 control response channel closed
Remote logging shutting down.Worker exited successfully!
21/06/02 00:12:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832 finished.
2021/06/02 00:12:43 Job state: DONE
21/06/02 00:12:43 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
21/06/02 00:12:43 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622592756036881929-jenkins-0602001239-b9a685d4_839fabdf-2016-4e3a-b1cc-f5de8cd50832
2021/06/02 00:12:43 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (30.79s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 30.817s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.023s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 19s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/bz5w6j3re3x44
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4747
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4747/display/redirect?page=changes>
Changes:
[zyichi] Minor fix to prebuilding sdk workflow timeout setting
[Ismaël Mejía] [BEAM-12423] Upgrade pyarrow to support version 4.0.0 too
------------------------------------------
[...truncated 545.59 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 18:15:05 Prepared job with id: go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8 and staging token: go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8
2021/06/01 18:15:05 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622571305291135607
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.go.
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.null.
21/06/01 18:15:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622571305288527188_61ef50fe-c7a1-433a-9ff9-df1854cf3ea8.
2021/06/01 18:15:08 Staged binary artifact with token:
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
21/06/01 18:15:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
2021/06/01 18:15:08 Submitted job: go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
2021/06/01 18:15:08 Job state: STOPPED
2021/06/01 18:15:08 Job state: STARTING
2021/06/01 18:15:08 Job state: RUNNING
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e on Spark master local
21/06/01 18:15:08 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 18:15:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e: Pipeline translated successfully. Computing outputs
21/06/01 18:15:09 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37539 ...
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37473 ...
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37345 ...
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3369206 ns
21/06/01 18:15:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1902441 ns
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/eaffad8902bcc2011b67bdfb4f2556d7fe2eddc5af547157766283b182fb692b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 18:15:10 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 18:15:10 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e837bbee9b0669bf36e667d628cb38df60ced3686c226a90a004c5b3775d7e4b. Logs:
2021/06/01 18:15:10 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622571305288527188"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37473"} artifact_endpoint:{url:"localhost:39421"} control_endpoint:{url:"localhost:37539"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/eaffad8902bcc2011b67bdfb4f2556d7fe2eddc5af547157766283b182fb692b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 18:15:10 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:38387
2021/06/01 18:15:10 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/06/01 18:15:10 control response channel closed
21/06/01 18:15:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e finished.
2021/06/01 18:15:11 Job state: DONE
21/06/01 18:15:11 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
21/06/01 18:15:11 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622571305288527188-jenkins-0601181508-7ed5b8dd_12aefc22-f2d7-40e2-a1ec-5210c952959e
2021/06/01 18:15:11 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (34.41s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 34.432s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 45s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/d6qpimrkcdspw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4746
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4746/display/redirect>
Changes:
------------------------------------------
[...truncated 545.77 KB...]
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 12:12:42 Prepared job with id: go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60 and staging token: go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60
2021/06/01 12:12:42 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622549562344721298
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.go.
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.null.
21/06/01 12:12:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622549562342099029_c09ccb1c-e2aa-45c1-be64-27b76923dc60.
2021/06/01 12:12:44 Staged binary artifact with token:
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
21/06/01 12:12:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
2021/06/01 12:12:44 Submitted job: go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
2021/06/01 12:12:44 Job state: STOPPED
2021/06/01 12:12:44 Job state: STARTING
2021/06/01 12:12:44 Job state: RUNNING
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d on Spark master local
21/06/01 12:12:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 12:12:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d: Pipeline translated successfully. Computing outputs
21/06/01 12:12:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45455 ...
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34893 ...
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37033 ...
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2641799 ns
21/06/01 12:12:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1475344 ns
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e5b872fdb855e253be8270f8ceb63d6ff27a375d99ddb8ea467820a268d156a8/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 12:12:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 12:12:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1b0c83e67f62948db6498889c9b2bfd06dfef7c80c5ca00562dba63fc6537254. Logs:
2021/06/01 12:12:46 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622549562342099029"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622549562342099029-jenkins-0601121244-88a2927a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34893"} artifact_endpoint:{url:"localhost:44485"} control_endpoint:{url:"localhost:45455"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e5b872fdb855e253be8270f8ceb63d6ff27a375d99ddb8ea467820a268d156a8/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 12:12:46 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40417
2021/06/01 12:12:46 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/01 12:12:46 control response channel closed
21/06/01 12:12:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d finished.
2021/06/01 12:12:47 Job state: DONE
21/06/01 12:12:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
21/06/01 12:12:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622549562342099029-jenkins-0601121244-88a2927a_2f9c993e-2b06-4766-82ab-2ccbb215a16d
2021/06/01 12:12:47 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.48s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.494s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 20s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
https://gradle.com/s/h7qjtxg4o3l4q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4745
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4745/display/redirect>
Changes:
------------------------------------------
[...truncated 546.35 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 06:13:17 Prepared job with id: go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208 and staging token: go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208
2021/06/01 06:13:17 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622527997923926760
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.go.
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.null.
21/06/01 06:13:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622527997921345339_e89469a2-f2d3-49db-a97c-a8addc242208.
2021/06/01 06:13:20 Staged binary artifact with token:
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
21/06/01 06:13:20 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
2021/06/01 06:13:20 Submitted job: go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
2021/06/01 06:13:20 Job state: STOPPED
2021/06/01 06:13:20 Job state: STARTING
2021/06/01 06:13:20 Job state: RUNNING
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1 on Spark master local
21/06/01 06:13:20 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 06:13:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1: Pipeline translated successfully. Computing outputs
21/06/01 06:13:20 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33429 ...
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33361 ...
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44779 ...
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2774368 ns
21/06/01 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1435005 ns
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/63906311f9221f387ca0b03c24aeb74d4c9e63eee4ddb305a5debafe612ca5a3/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 06:13:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 06:13:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 414b39292ab02f1b521b8c4c3d45293b6d78fc01f540e1ade8d048059e95a86b. Logs:
2021/06/01 06:13:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622527997921345339"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622527997921345339-jenkins-0601061320-d7855d32"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33361"} artifact_endpoint:{url:"localhost:35153"} control_endpoint:{url:"localhost:33429"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/63906311f9221f387ca0b03c24aeb74d4c9e63eee4ddb305a5debafe612ca5a3/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 06:13:21 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40595
2021/06/01 06:13:21 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/01 06:13:21 control response channel closed
21/06/01 06:13:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1 finished.
2021/06/01 06:13:22 Job state: DONE
21/06/01 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
21/06/01 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622527997921345339-jenkins-0601061320-d7855d32_c518d929-a27a-4a1b-8336-843f5a15d5a1
2021/06/01 06:13:22 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (23.49s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 23.506s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 56s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/mkj4jbn75oxcc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4744
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4744/display/redirect>
Changes:
------------------------------------------
[...truncated 545.89 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/06/01 00:13:07 Prepared job with id: go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c and staging token: go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c
2021/06/01 00:13:07 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622506387941605896
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.go.
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.null.
21/06/01 00:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622506387939373424_87548c4a-f103-48f4-8793-28c7d1fbf48c.
2021/06/01 00:13:10 Staged binary artifact with token:
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
21/06/01 00:13:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
2021/06/01 00:13:10 Submitted job: go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
2021/06/01 00:13:10 Job state: STOPPED
2021/06/01 00:13:10 Job state: STARTING
2021/06/01 00:13:10 Job state: RUNNING
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc on Spark master local
21/06/01 00:13:10 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/06/01 00:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc: Pipeline translated successfully. Computing outputs
21/06/01 00:13:10 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43579 ...
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45393 ...
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35371 ...
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3036448 ns
21/06/01 00:13:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1769681 ns
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/8c93a6c62be388cb97491ee6aa433985d8f1bbc3adfbe56c420feaca49831817/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/06/01 00:13:11 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/06/01 00:13:11 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container df085ae8fa951f02f3f03199c793fcdf6fd37d78c7c497321e9cfbc8f2e7d499. Logs:
2021/06/01 00:13:11 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622506387939373424"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622506387939373424-jenkins-0601001310-fbb4149a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45393"} artifact_endpoint:{url:"localhost:44509"} control_endpoint:{url:"localhost:43579"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/8c93a6c62be388cb97491ee6aa433985d8f1bbc3adfbe56c420feaca49831817/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/06/01 00:13:11 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:46695
2021/06/01 00:13:11 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/06/01 00:13:11 control response channel closed
21/06/01 00:13:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc finished.
2021/06/01 00:13:12 Job state: DONE
21/06/01 00:13:12 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
21/06/01 00:13:12 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622506387939373424-jenkins-0601001310-fbb4149a_2908311d-351f-4e27-9403-caa9f2391acc
2021/06/01 00:13:12 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (23.54s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 23.564s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 47s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/izasu2jlisqai
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4743
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4743/display/redirect>
Changes:
------------------------------------------
[...truncated 545.94 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 18:12:44 Prepared job with id: go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44 and staging token: go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44
2021/05/31 18:12:44 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622484764975225006
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.go.
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.null.
21/05/31 18:12:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622484764972370019_b371a5ee-e566-41f0-a241-bfa7205bce44.
2021/05/31 18:12:47 Staged binary artifact with token:
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
21/05/31 18:12:47 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
2021/05/31 18:12:47 Submitted job: go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
2021/05/31 18:12:47 Job state: STOPPED
2021/05/31 18:12:47 Job state: STARTING
2021/05/31 18:12:47 Job state: RUNNING
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255 on Spark master local
21/05/31 18:12:47 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 18:12:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255: Pipeline translated successfully. Computing outputs
21/05/31 18:12:48 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35419 ...
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46655 ...
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46337 ...
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2933349 ns
21/05/31 18:12:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1635793 ns
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/76a835c81f77ba8d4d538a30e678d2727424d1f37c160c1a33885a65c660a02b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 18:12:49 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 18:12:49 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d3889c99a3722b2e6095a3135eecced6c3251f479bc23e4010794763782667c5. Logs:
2021/05/31 18:12:48 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622484764972370019"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622484764972370019-jenkins-0531181247-6f9350b7"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46655"} artifact_endpoint:{url:"localhost:37755"} control_endpoint:{url:"localhost:35419"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/76a835c81f77ba8d4d538a30e678d2727424d1f37c160c1a33885a65c660a02b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 18:12:48 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44417
2021/05/31 18:12:49 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/31 18:12:49 control response channel closed
21/05/31 18:12:49 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255 finished.
2021/05/31 18:12:49 Job state: DONE
21/05/31 18:12:49 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
21/05/31 18:12:49 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622484764972370019-jenkins-0531181247-6f9350b7_1e9fe96f-d082-4afd-adf7-5150dfe83255
2021/05/31 18:12:49 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.47s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.491s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 23s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/ti6spzi4uzars
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4742
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4742/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12277] Update Flink 1.13 version to 1.13.1
[Ismaël Mejía] [BEAM-12424] Update Flink 1.12 to version 1.12.4
------------------------------------------
[...truncated 545.11 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 12:12:12 Prepared job with id: go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a and staging token: go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a
2021/05/31 12:12:12 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622463132233630280
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.go.
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.null.
21/05/31 12:12:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622463132231234252_87ea7f69-b8a8-4965-9cfc-ecf7e66b018a.
2021/05/31 12:12:14 Staged binary artifact with token:
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
21/05/31 12:12:14 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
2021/05/31 12:12:14 Submitted job: go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
2021/05/31 12:12:14 Job state: STOPPED
2021/05/31 12:12:14 Job state: STARTING
2021/05/31 12:12:14 Job state: RUNNING
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1 on Spark master local
21/05/31 12:12:14 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 12:12:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1: Pipeline translated successfully. Computing outputs
21/05/31 12:12:15 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45521 ...
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37107 ...
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37695 ...
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2542551 ns
21/05/31 12:12:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1420436 ns
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/7f8e7e88a5e9e62c99e16b2a15d24de168a48d59695f56982e342efeddfce32b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 12:12:16 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 12:12:16 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2ef7d796de5a91fcedc9ad5f14e0111c742c2050f5ddc6a275e6d4d61d5069a8. Logs:
2021/05/31 12:12:16 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622463132231234252"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622463132231234252-jenkins-0531121214-10387377"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37107"} artifact_endpoint:{url:"localhost:43953"} control_endpoint:{url:"localhost:45521"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/7f8e7e88a5e9e62c99e16b2a15d24de168a48d59695f56982e342efeddfce32b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 12:12:16 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44379
2021/05/31 12:12:16 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/31 12:12:16 control response channel closed
21/05/31 12:12:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1 finished.
2021/05/31 12:12:17 Job state: DONE
21/05/31 12:12:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
21/05/31 12:12:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622463132231234252-jenkins-0531121214-10387377_1fe87048-f367-475e-a24e-87e05dcdd1d1
2021/05/31 12:12:17 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.41s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.426s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 53s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/5kql55mznbpvm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4741
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4741/display/redirect>
Changes:
------------------------------------------
[...truncated 546.67 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 06:13:08 Prepared job with id: go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23 and staging token: go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23
2021/05/31 06:13:08 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622441588542188274
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.go.
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.null.
21/05/31 06:13:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622441588539781408_82fa32db-7b47-4b0c-9c49-6d982c135e23.
2021/05/31 06:13:10 Staged binary artifact with token:
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
21/05/31 06:13:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
2021/05/31 06:13:10 Submitted job: go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
2021/05/31 06:13:10 Job state: STOPPED
2021/05/31 06:13:10 Job state: STARTING
2021/05/31 06:13:10 Job state: RUNNING
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0 on Spark master local
21/05/31 06:13:10 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 06:13:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0: Pipeline translated successfully. Computing outputs
21/05/31 06:13:11 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42597 ...
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43983 ...
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37435 ...
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2884294 ns
21/05/31 06:13:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1502687 ns
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/bfc4a8156e0981dc9ade400d1fa2c499f57b9636545a7a306392b8640c74dc6b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 06:13:12 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 06:13:12 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 02566e585ffa101d4275a014b7e2447ee124450ed0983f50a203962a7cc336c2. Logs:
2021/05/31 06:13:12 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622441588539781408"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622441588539781408-jenkins-0531061310-24d36646"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43983"} artifact_endpoint:{url:"localhost:41691"} control_endpoint:{url:"localhost:42597"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/bfc4a8156e0981dc9ade400d1fa2c499f57b9636545a7a306392b8640c74dc6b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 06:13:12 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:42061
2021/05/31 06:13:12 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/31 06:13:12 control response channel closed
21/05/31 06:13:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0 finished.
2021/05/31 06:13:13 Job state: DONE
21/05/31 06:13:13 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
21/05/31 06:13:13 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622441588539781408-jenkins-0531061310-24d36646_d16983ed-5aaf-4c7c-a89f-260be62e7db0
2021/05/31 06:13:13 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (23.57s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 23.585s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 45s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/rpdp6zxd7duag
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4740
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4740/display/redirect>
Changes:
------------------------------------------
[...truncated 546.15 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/31 00:12:59 Prepared job with id: go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd and staging token: go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd
2021/05/31 00:12:59 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622419979457710193
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.go.
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.null.
21/05/31 00:13:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622419979455082976_f2d3d44f-3e05-44a8-9ffd-8453ff63a1bd.
2021/05/31 00:13:01 Staged binary artifact with token:
21/05/31 00:13:01 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
21/05/31 00:13:01 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
2021/05/31 00:13:01 Submitted job: go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
2021/05/31 00:13:01 Job state: STOPPED
2021/05/31 00:13:01 Job state: STARTING
2021/05/31 00:13:01 Job state: RUNNING
21/05/31 00:13:01 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/31 00:13:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7 on Spark master local
21/05/31 00:13:02 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/31 00:13:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7: Pipeline translated successfully. Computing outputs
21/05/31 00:13:02 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40333 ...
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36149 ...
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44273 ...
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2712384 ns
21/05/31 00:13:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1521661 ns
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/277eee75898a037c1773cfacebd0ab3f97df4df93c7dc1db3c74806dbc35bfc9/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/31 00:13:03 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/31 00:13:03 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ffcf8e75a0472930a5a986fc0780f19d1f7817e628790fc366fda32219218b50. Logs:
2021/05/31 00:13:03 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622419979455082976"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622419979455082976-jenkins-0531001301-76477bbb"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36149"} artifact_endpoint:{url:"localhost:33525"} control_endpoint:{url:"localhost:40333"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/277eee75898a037c1773cfacebd0ab3f97df4df93c7dc1db3c74806dbc35bfc9/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/31 00:13:03 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:36863
2021/05/31 00:13:03 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
2021/05/31 00:13:03 control response channel closed
Worker exited successfully!
21/05/31 00:13:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7 finished.
2021/05/31 00:13:04 Job state: DONE
21/05/31 00:13:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
21/05/31 00:13:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622419979455082976-jenkins-0531001301-76477bbb_4eaee7dd-74a5-4289-bada-0535d2422aa7
2021/05/31 00:13:04 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.29s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.307s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.017s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 39s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/o6iku4jxuupb2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4739
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4739/display/redirect>
Changes:
------------------------------------------
[...truncated 546.10 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 18:13:41 Prepared job with id: go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749 and staging token: go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749
2021/05/30 18:13:41 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622398421600806877
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.go.
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.null.
21/05/30 18:13:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622398421598058738_4ea92fdc-74d3-4765-a92c-f42da3fe9749.
2021/05/30 18:13:44 Staged binary artifact with token:
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
21/05/30 18:13:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
2021/05/30 18:13:44 Submitted job: go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
2021/05/30 18:13:44 Job state: STOPPED
2021/05/30 18:13:44 Job state: STARTING
2021/05/30 18:13:44 Job state: RUNNING
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3 on Spark master local
21/05/30 18:13:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 18:13:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3: Pipeline translated successfully. Computing outputs
21/05/30 18:13:44 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39137 ...
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43715 ...
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46565 ...
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3094222 ns
21/05/30 18:13:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1595540 ns
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/5feb5e47e94346187ea5671698aff984dff21d933cbfd4d81a5f4c2fbbeffc29/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 18:13:45 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 18:13:45 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2d0a11b7f47a743f3c43c6982512968ad7fd50ff95df35aaae9a49db1610f0f6. Logs:
2021/05/30 18:13:45 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622398421598058738"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622398421598058738-jenkins-0530181344-c9f797d2"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43715"} artifact_endpoint:{url:"localhost:46505"} control_endpoint:{url:"localhost:39137"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/5feb5e47e94346187ea5671698aff984dff21d933cbfd4d81a5f4c2fbbeffc29/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 18:13:45 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:41675
2021/05/30 18:13:45 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 18:13:45 control response channel closed
21/05/30 18:13:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3 finished.
2021/05/30 18:13:46 Job state: DONE
21/05/30 18:13:46 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
21/05/30 18:13:46 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622398421598058738-jenkins-0530181344-c9f797d2_36d4727d-6a13-4183-8f9c-6ad6a40cc8a3
2021/05/30 18:13:46 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.38s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.402s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 20s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/67hiqbn37wkqa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4738
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4738/display/redirect>
Changes:
------------------------------------------
[...truncated 545.60 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 12:18:38 Prepared job with id: go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7 and staging token: go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7
2021/05/30 12:18:38 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622377118029096917
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.go.
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.null.
21/05/30 12:18:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622377118021005828_7b1f8472-9cee-4daf-ad8a-754445e94fb7.
21/05/30 12:18:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
21/05/30 12:18:43 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
21/05/30 12:18:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
2021/05/30 12:18:43 Staged binary artifact with token:
2021/05/30 12:18:43 Submitted job: go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
2021/05/30 12:18:43 Job state: STOPPED
2021/05/30 12:18:43 Job state: STARTING
2021/05/30 12:18:43 Job state: RUNNING
21/05/30 12:18:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12 on Spark master local
21/05/30 12:18:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 12:18:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12: Pipeline translated successfully. Computing outputs
21/05/30 12:18:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 12:18:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 12:18:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 12:18:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36521 ...
21/05/30 12:18:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44869 ...
21/05/30 12:18:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 12:18:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44005 ...
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 12:18:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 10570479 ns
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/681d13130eee1d237929b749af12670a818b583834523270f8f417f1052bb9de/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 12:18:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 7593719 ns
21/05/30 12:18:47 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 12:18:47 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c16b378ca7fc2609daed50fd8f4aa2264288820c7588eb352558816e965fb069. Logs:
2021/05/30 12:18:46 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622377118021005828"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622377118021005828-jenkins-0530121843-2eb720dc"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44869"} artifact_endpoint:{url:"localhost:44871"} control_endpoint:{url:"localhost:36521"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/681d13130eee1d237929b749af12670a818b583834523270f8f417f1052bb9de/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 12:18:46 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:33655
2021/05/30 12:18:46 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 12:18:47 control response channel closed
21/05/30 12:18:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12 finished.
2021/05/30 12:18:48 Job state: DONE
21/05/30 12:18:48 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
21/05/30 12:18:48 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622377118021005828-jenkins-0530121843-2eb720dc_a1311ada-7fda-4d86-9875-c5aa9c816c12
2021/05/30 12:18:48 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (58.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 58.327s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.059s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 18m 27s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/mfsilndttcdtw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4737
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4737/display/redirect>
Changes:
------------------------------------------
[...truncated 546.12 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 06:13:15 Prepared job with id: go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214 and staging token: go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214
2021/05/30 06:13:15 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622355195971824488
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.go.
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.null.
21/05/30 06:13:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622355195968983383_d1c1d493-ed92-4ace-9ee9-09cf2d00c214.
2021/05/30 06:13:18 Staged binary artifact with token:
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
21/05/30 06:13:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
2021/05/30 06:13:18 Submitted job: go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
2021/05/30 06:13:18 Job state: STOPPED
2021/05/30 06:13:18 Job state: STARTING
2021/05/30 06:13:18 Job state: RUNNING
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98 on Spark master local
21/05/30 06:13:18 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 06:13:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98: Pipeline translated successfully. Computing outputs
21/05/30 06:13:19 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44645 ...
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37447 ...
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38223 ...
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3258757 ns
21/05/30 06:13:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1694343 ns
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/2d89b203547beb73d947a2b3290d5c251a09239dce771c5bcfc3940d4eef4a07/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 06:13:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 06:13:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f57ab0038d27c600a5595556044bddd1fbd9fda75914b7451fc41f1be2fd8f5e. Logs:
2021/05/30 06:13:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622355195968983383"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622355195968983383-jenkins-0530061318-18c67c42"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37447"} artifact_endpoint:{url:"localhost:34509"} control_endpoint:{url:"localhost:44645"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/2d89b203547beb73d947a2b3290d5c251a09239dce771c5bcfc3940d4eef4a07/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 06:13:21 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:41087
2021/05/30 06:13:21 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 06:13:21 control response channel closed
21/05/30 06:13:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98 finished.
2021/05/30 06:13:22 Job state: DONE
21/05/30 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
21/05/30 06:13:22 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622355195968983383-jenkins-0530061318-18c67c42_42f61f08-54d7-4b66-83de-87243bde3c98
2021/05/30 06:13:22 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (30.26s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 30.282s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 42s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/ih4amcevi5xiw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4736
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4736/display/redirect>
Changes:
------------------------------------------
[...truncated 546.02 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/30 00:13:18 Prepared job with id: go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70 and staging token: go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70
2021/05/30 00:13:18 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622333598502582764
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.go.
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.null.
21/05/30 00:13:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622333598500075722_2b387338-b48b-4995-a817-4fa87e114f70.
2021/05/30 00:13:21 Staged binary artifact with token:
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
21/05/30 00:13:21 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
2021/05/30 00:13:21 Submitted job: go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
2021/05/30 00:13:21 Job state: STOPPED
2021/05/30 00:13:21 Job state: STARTING
2021/05/30 00:13:21 Job state: RUNNING
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430 on Spark master local
21/05/30 00:13:21 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/30 00:13:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430: Pipeline translated successfully. Computing outputs
21/05/30 00:13:21 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46515 ...
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38367 ...
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45473 ...
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2971495 ns
21/05/30 00:13:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1645436 ns
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/6ba176465a32f401c71285dccaa6c6dd39b8ee3cc7080f107dbffe2d6e77567a/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/30 00:13:22 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/30 00:13:22 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container afb48784e97f657a52ad32364c5b4aa6205093a3203f865951057f14579985f8. Logs:
2021/05/30 00:13:22 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622333598500075722"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622333598500075722-jenkins-0530001321-bf8d8332"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38367"} artifact_endpoint:{url:"localhost:38393"} control_endpoint:{url:"localhost:46515"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/6ba176465a32f401c71285dccaa6c6dd39b8ee3cc7080f107dbffe2d6e77567a/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/30 00:13:22 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:37733
2021/05/30 00:13:22 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/30 00:13:22 control response channel closed
21/05/30 00:13:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430 finished.
2021/05/30 00:13:23 Job state: DONE
21/05/30 00:13:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
21/05/30 00:13:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622333598500075722-jenkins-0530001321-bf8d8332_2fa3625e-f5f1-4293-bb8d-0c1277844430
2021/05/30 00:13:23 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (24.87s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 24.896s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 2s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/vhhbpfpihk5ee
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4735
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4735/display/redirect>
Changes:
------------------------------------------
[...truncated 546.34 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 18:13:40 Prepared job with id: go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd and staging token: go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd
2021/05/29 18:13:40 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622312020339574884
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.go.
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.null.
21/05/29 18:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622312020336805684_4faf89ad-1865-42e3-b8ef-b2267a34e7dd.
2021/05/29 18:13:42 Staged binary artifact with token:
21/05/29 18:13:42 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
21/05/29 18:13:42 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
2021/05/29 18:13:43 Submitted job: go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
2021/05/29 18:13:43 Job state: STOPPED
2021/05/29 18:13:43 Job state: STARTING
2021/05/29 18:13:43 Job state: RUNNING
21/05/29 18:13:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/29 18:13:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed on Spark master local
21/05/29 18:13:43 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 18:13:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed: Pipeline translated successfully. Computing outputs
21/05/29 18:13:43 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38779 ...
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35917 ...
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44107 ...
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2966289 ns
21/05/29 18:13:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1634259 ns
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e4698c1ccb4f6966ff1f812b3d650c3bcd35acaf438e9cf31b668c6817bdfa39/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 18:13:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 18:13:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 08b7423ed2d2e4661fa3fa9a2f1970ddd02980f30f118d587ce7e397d84ffec4. Logs:
2021/05/29 18:13:44 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622312020336805684"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622312020336805684-jenkins-0529181342-bc8292f8"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35917"} artifact_endpoint:{url:"localhost:38127"} control_endpoint:{url:"localhost:38779"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e4698c1ccb4f6966ff1f812b3d650c3bcd35acaf438e9cf31b668c6817bdfa39/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 18:13:44 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:45931
2021/05/29 18:13:44 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/29 18:13:44 control response channel closed
21/05/29 18:13:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed finished.
2021/05/29 18:13:45 Job state: DONE
21/05/29 18:13:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
21/05/29 18:13:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622312020336805684-jenkins-0529181342-bc8292f8_02b297e7-bfd7-4373-984a-f47fbdbff8ed
2021/05/29 18:13:45 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.45s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.473s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 19s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/vi5mb6asmllaq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4734
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4734/display/redirect>
Changes:
------------------------------------------
[...truncated 545.87 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 12:10:58 Prepared job with id: go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348 and staging token: go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348
2021/05/29 12:10:58 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622290258006291245
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.go.
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.null.
21/05/29 12:11:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622290258002805754_95af613c-a72c-462f-9ac1-1ad5b4d14348.
2021/05/29 12:11:00 Staged binary artifact with token:
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
21/05/29 12:11:00 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
2021/05/29 12:11:00 Submitted job: go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
2021/05/29 12:11:00 Job state: STOPPED
2021/05/29 12:11:00 Job state: STARTING
2021/05/29 12:11:00 Job state: RUNNING
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11 on Spark master local
21/05/29 12:11:00 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 12:11:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11: Pipeline translated successfully. Computing outputs
21/05/29 12:11:01 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38499 ...
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44359 ...
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35419 ...
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3291161 ns
21/05/29 12:11:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1894607 ns
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/69f8d509ef0235415bac27f52fbe3866b76c883fe650a38dc75109732b5391aa/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 12:11:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 12:11:02 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 70226da8d782d00608f23c34b12e4588c164abd81ecd5a5f105121fb9c62ccd4. Logs:
2021/05/29 12:11:01 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622290258002805754"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622290258002805754-jenkins-0529121100-1cff0ae2"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44359"} artifact_endpoint:{url:"localhost:46799"} control_endpoint:{url:"localhost:38499"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/69f8d509ef0235415bac27f52fbe3866b76c883fe650a38dc75109732b5391aa/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 12:11:01 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:43855
2021/05/29 12:11:02 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/29 12:11:02 control response channel closed
21/05/29 12:11:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11 finished.
2021/05/29 12:11:03 Job state: DONE
21/05/29 12:11:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
21/05/29 12:11:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622290258002805754-jenkins-0529121100-1cff0ae2_85aabff5-0ce6-4a12-ab5d-11d12550eb11
2021/05/29 12:11:03 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (29.16s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 29.183s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 34s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/g7ikxeg5jabbq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4733
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4733/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11608] Development environment set up automation (#14584)
------------------------------------------
[...truncated 546.47 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 06:13:51 Prepared job with id: go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee and staging token: go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee
2021/05/29 06:13:51 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622268831858653583
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.go.
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.null.
21/05/29 06:13:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622268831856323278_4dc2da95-e73f-4e19-b69c-d566e66b2dee.
2021/05/29 06:13:54 Staged binary artifact with token:
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
21/05/29 06:13:54 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
2021/05/29 06:13:54 Submitted job: go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
2021/05/29 06:13:54 Job state: STOPPED
2021/05/29 06:13:54 Job state: STARTING
2021/05/29 06:13:54 Job state: RUNNING
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083 on Spark master local
21/05/29 06:13:54 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 06:13:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083: Pipeline translated successfully. Computing outputs
21/05/29 06:13:54 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44555 ...
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42063 ...
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40507 ...
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2719116 ns
21/05/29 06:13:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1501200 ns
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/f9819db2fd14a32a59cb70a1f08d6efc402dafad294b62e990f2761622b53d87/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 06:13:55 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 06:13:55 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b8b137c76d97cce61aa2300b66bf9eeda1d76a6d55da395b48b1b96998ec86be. Logs:
2021/05/29 06:13:55 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622268831856323278"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622268831856323278-jenkins-0529061354-601e2e5"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42063"} artifact_endpoint:{url:"localhost:40261"} control_endpoint:{url:"localhost:44555"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/f9819db2fd14a32a59cb70a1f08d6efc402dafad294b62e990f2761622b53d87/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 06:13:55 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44621
2021/05/29 06:13:55 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
2021/05/29 06:13:55 control response channel closed
Worker exited successfully!
21/05/29 06:13:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083 finished.
2021/05/29 06:13:56 Job state: DONE
21/05/29 06:13:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
21/05/29 06:13:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622268831856323278-jenkins-0529061354-601e2e5_0b4c7649-0dad-471a-bd62-c90f9d239083
2021/05/29 06:13:56 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.09s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.111s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 29s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/nleup2fltcxlw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4732
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4732/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11988] added a fix for missing sample.txt (#14889)
[noreply] [BEAM-621] Add MapKeys, MapValues transforms (#14273)
------------------------------------------
[...truncated 547.69 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/29 00:17:42 Prepared job with id: go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3 and staging token: go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3
2021/05/29 00:17:42 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622247462353935115
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.go.
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.null.
21/05/29 00:17:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622247462350129770_0ab13041-0beb-4be1-b4d5-7379779e75e3.
2021/05/29 00:17:48 Staged binary artifact with token:
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
21/05/29 00:17:48 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
2021/05/29 00:17:48 Submitted job: go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
2021/05/29 00:17:48 Job state: STOPPED
2021/05/29 00:17:48 Job state: STARTING
2021/05/29 00:17:48 Job state: RUNNING
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1 on Spark master local
21/05/29 00:17:48 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/29 00:17:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1: Pipeline translated successfully. Computing outputs
21/05/29 00:17:49 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33633 ...
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37227 ...
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40735 ...
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 4823301 ns
21/05/29 00:17:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2646354 ns
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/1593fbe8e99ce1cb89a692e92d95acb14f6f105d25ca5bfa62774ba15a1bcbde/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/29 00:17:51 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/29 00:17:51 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container eaa36c661592ea9c019f5d285bcad1285ffde606de24b7cee4c827b814bc4f1d. Logs:
2021/05/29 00:17:50 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622247462350129770"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622247462350129770-jenkins-0529001748-b15eb165"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37227"} artifact_endpoint:{url:"localhost:33175"} control_endpoint:{url:"localhost:33633"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/1593fbe8e99ce1cb89a692e92d95acb14f6f105d25ca5bfa62774ba15a1bcbde/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/29 00:17:50 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:43787
2021/05/29 00:17:50 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/29 00:17:51 control response channel closed
21/05/29 00:17:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1 finished.
2021/05/29 00:17:53 Job state: DONE
21/05/29 00:17:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
21/05/29 00:17:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622247462350129770-jenkins-0529001748-b15eb165_1cc8640b-c21b-4d11-bd29-2e20f034f3f1
2021/05/29 00:17:53 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (41.97s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 41.996s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 17m 29s
79 actionable tasks: 62 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/hliyefdkupi2w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4731
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4731/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12277] Add flink 1.13 build target.
[Ismaël Mejía] [BEAM-12277] Make Flink 1.13 the default Flink version to run the tests
[Ismaël Mejía] [BEAM-12277] Update expected metric name formatting.
[Ismaël Mejía] [BEAM-12277] fix mistake in source overrides
------------------------------------------
[...truncated 547.22 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 18:16:51 Prepared job with id: go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753 and staging token: go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753
2021/05/28 18:16:51 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622225811470283233
21/05/28 18:16:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.
21/05/28 18:16:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.go.
21/05/28 18:16:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.null.
21/05/28 18:16:54 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622225811467135324_f9953492-101a-4bc9-aff3-1cc6c1803753.
2021/05/28 18:16:54 Staged binary artifact with token:
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
21/05/28 18:16:54 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
2021/05/28 18:16:54 Submitted job: go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
2021/05/28 18:16:54 Job state: STOPPED
2021/05/28 18:16:54 Job state: STARTING
2021/05/28 18:16:54 Job state: RUNNING
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb on Spark master local
21/05/28 18:16:54 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 18:16:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb: Pipeline translated successfully. Computing outputs
21/05/28 18:16:54 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37135 ...
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37455 ...
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35795 ...
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3006785 ns
21/05/28 18:16:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1593739 ns
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/98fd382c021abc3ae4904aa88b522d4143ea96511afec468a0dfa42ca7d26855/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 18:16:55 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 18:16:55 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 20db6c0acab9e291538d4dc45c057d109ce8f158b505a92aca468a3153b961d6. Logs:
2021/05/28 18:16:55 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622225811467135324"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622225811467135324-jenkins-0528181654-e74bff0d"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37455"} artifact_endpoint:{url:"localhost:35115"} control_endpoint:{url:"localhost:37135"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/98fd382c021abc3ae4904aa88b522d4143ea96511afec468a0dfa42ca7d26855/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 18:16:55 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:38225
2021/05/28 18:16:55 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/28 18:16:55 control response channel closed
21/05/28 18:16:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb finished.
2021/05/28 18:16:56 Job state: DONE
21/05/28 18:16:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
21/05/28 18:16:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622225811467135324-jenkins-0528181654-e74bff0d_f9b5c278-4472-4f9b-a954-790e21d56dfb
2021/05/28 18:16:56 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.30s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.324s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 28s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/szajnpq3vmsa4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4730
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4730/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12384] Refine generic types on Read.Bounded internals
[Ismaël Mejía] [BEAM-12384] Set output typeDescriptor explictly in Read.Bounded
[Ismaël Mejía] [BEAM-12384] Infer typeDescriptor from coder if typeDescriptor is not
[Ismaël Mejía] [BEAM-12415] Update Spark 3 version to 3.1.2
------------------------------------------
[...truncated 548.35 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 12:22:12 Prepared job with id: go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521 and staging token: go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521
2021/05/28 12:22:12 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622204532388027149
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.go.
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.null.
21/05/28 12:22:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622204532385149167_f7673c53-ebb5-4d7c-ae72-2daa22482521.
2021/05/28 12:22:15 Staged binary artifact with token:
21/05/28 12:22:15 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
21/05/28 12:22:15 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
2021/05/28 12:22:15 Submitted job: go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
2021/05/28 12:22:15 Job state: STOPPED
2021/05/28 12:22:15 Job state: STARTING
2021/05/28 12:22:15 Job state: RUNNING
21/05/28 12:22:15 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 12:22:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48 on Spark master local
21/05/28 12:22:16 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 12:22:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48: Pipeline translated successfully. Computing outputs
21/05/28 12:22:16 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46197 ...
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40195 ...
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40595 ...
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3179692 ns
21/05/28 12:22:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1642072 ns
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/84d2ae7678f1781c628b9a838b95fd94e72e698a7e8e8012ec50764e1cfe32bb/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 12:22:18 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 12:22:18 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 5f212d1ccc1582ded45240c3894931584b9fac14642746385d8632cb64cd3fba. Logs:
2021/05/28 12:22:17 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622204532385149167"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622204532385149167-jenkins-0528122215-6d46ca07"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40195"} artifact_endpoint:{url:"localhost:45643"} control_endpoint:{url:"localhost:46197"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/84d2ae7678f1781c628b9a838b95fd94e72e698a7e8e8012ec50764e1cfe32bb/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 12:22:17 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40707
2021/05/28 12:22:18 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/28 12:22:18 control response channel closed
21/05/28 12:22:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48 finished.
2021/05/28 12:22:19 Job state: DONE
21/05/28 12:22:19 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
21/05/28 12:22:19 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622204532385149167-jenkins-0528122215-6d46ca07_1336f38c-a572-4b30-81c3-9722e90c9b48
2021/05/28 12:22:19 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (33.23s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 33.254s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.016s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 21m 55s
79 actionable tasks: 69 executed, 10 from cache
Publishing build scan...
https://gradle.com/s/myizweq6jf72w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4729
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4729/display/redirect?page=changes>
Changes:
[Boyuan Zhang] Populate use_sdf_read in ExpansionService.
------------------------------------------
[...truncated 546.66 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 06:16:15 Prepared job with id: go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4 and staging token: go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4
2021/05/28 06:16:15 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622182575257977391
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.go.
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.null.
21/05/28 06:16:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622182575254318625_3a278f98-1859-4d3b-85b4-97efd17b2da4.
2021/05/28 06:16:17 Staged binary artifact with token:
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
21/05/28 06:16:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
2021/05/28 06:16:18 Submitted job: go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
2021/05/28 06:16:18 Job state: STOPPED
2021/05/28 06:16:18 Job state: STARTING
2021/05/28 06:16:18 Job state: RUNNING
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f on Spark master local
21/05/28 06:16:18 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 06:16:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f: Pipeline translated successfully. Computing outputs
21/05/28 06:16:18 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43305 ...
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34709 ...
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42331 ...
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3325343 ns
21/05/28 06:16:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1795198 ns
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/ea46141e183d214dd0c476f3efcaf464a89cceb21fa30390daece879e82d2624/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 06:16:19 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 06:16:19 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e21ebcb6bd63d9c786feb5ded9457b71869e28d3655b28c65556a899f8578be9. Logs:
2021/05/28 06:16:19 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622182575254318625"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622182575254318625-jenkins-0528061618-870d093"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34709"} artifact_endpoint:{url:"localhost:33397"} control_endpoint:{url:"localhost:43305"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/ea46141e183d214dd0c476f3efcaf464a89cceb21fa30390daece879e82d2624/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 06:16:19 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:37321
2021/05/28 06:16:19 Downloaded: /tmp/staged/worker (sha256: e8864cc86a6c584c6627052c274d26fcf22d42048a497895ae25198de3b228c8, size: 27884796)
Worker exited successfully!
2021/05/28 06:16:19 control response channel closed
21/05/28 06:16:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f finished.
2021/05/28 06:16:20 Job state: DONE
21/05/28 06:16:20 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
21/05/28 06:16:20 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622182575254318625-jenkins-0528061618-870d093_ad265be0-2e27-4909-aa35-3d1491b1466f
2021/05/28 06:16:20 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.96s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.977s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 15m 55s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/n2x5fizbjn2iq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4728
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4728/display/redirect?page=changes>
Changes:
[zhoufek] [BEAM-9487] Add trigger safety check to GroupByKey
[noreply] [BEAM-12370] Support side input in Samza portable runner (#14883)
[noreply] [BEAM-12158][BEAM-12341] Pubsub fixes. (#14896)
------------------------------------------
[...truncated 548.04 KB...]
component_coder_ids: "c9"
>
>
coders: <
key: "c2"
value: <
spec: <
urn: "beam:coder:length_prefix:v1"
>
component_coder_ids: "c1"
>
>
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/28 00:17:13 Prepared job with id: go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c and staging token: go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c
2021/05/28 00:17:13 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622161033648780657
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.go.
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.null.
21/05/28 00:17:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622161033644978888_1d3f3887-03a9-4c86-b902-c4d0b3d96a7c.
2021/05/28 00:17:18 Staged binary artifact with token:
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
21/05/28 00:17:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
2021/05/28 00:17:18 Submitted job: go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
2021/05/28 00:17:18 Job state: STOPPED
2021/05/28 00:17:18 Job state: STARTING
2021/05/28 00:17:18 Job state: RUNNING
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22 on Spark master local
21/05/28 00:17:18 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/28 00:17:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22: Pipeline translated successfully. Computing outputs
21/05/28 00:17:19 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45963 ...
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:41357 ...
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43751 ...
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3618762 ns
21/05/28 00:17:20 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2029276 ns
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/774f85400719e8b76c5eb8eeda0e183042ae29f0ff07ca1434cf6f64cbdbdd9d/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/28 00:17:20 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/28 00:17:20 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6e40c636e7a3c7c510deb8741d4f788400f17e4d9e3845002b20ac02bf30c2d8. Logs:
2021/05/28 00:17:20 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622161033644978888"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622161033644978888-jenkins-0528001718-767617c4"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41357"} artifact_endpoint:{url:"localhost:40391"} control_endpoint:{url:"localhost:45963"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/774f85400719e8b76c5eb8eeda0e183042ae29f0ff07ca1434cf6f64cbdbdd9d/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/28 00:17:20 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:45179
2021/05/28 00:17:20 Downloaded: /tmp/staged/worker (sha256: c2916509ce24fd164a6da497ba571012495647f3bed7500cdc24643eb4af3af8, size: 27884796)
Worker exited successfully!
2021/05/28 00:17:20 control response channel closed
21/05/28 00:17:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22 finished.
2021/05/28 00:17:21 Job state: DONE
21/05/28 00:17:21 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
21/05/28 00:17:21 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622161033644978888-jenkins-0528001718-767617c4_4a0583ba-6d5c-4bf5-b225-e21a2b47cd22
2021/05/28 00:17:21 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (41.20s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 41.228s
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.022s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 57s
79 actionable tasks: 58 executed, 21 from cache
Publishing build scan...
https://gradle.com/s/zfvgk3h62pf6m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4727
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4727/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12093] Overhaul ElasticsearchIO.Write (#14347)
------------------------------------------
[...truncated 564.52 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 18:14:51 Prepared job with id: go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d and staging token: go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d
2021/05/27 18:14:51 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622139291208428050
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.go.
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.null.
21/05/27 18:14:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622139291205489057_a17e8eba-43c0-4a75-b037-ef3c4bb2560d.
2021/05/27 18:14:53 Staged binary artifact with token:
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
21/05/27 18:14:53 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
2021/05/27 18:14:53 Submitted job: go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
2021/05/27 18:14:53 Job state: STOPPED
2021/05/27 18:14:53 Job state: STARTING
2021/05/27 18:14:53 Job state: RUNNING
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef on Spark master local
21/05/27 18:14:53 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 18:14:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef: Pipeline translated successfully. Computing outputs
21/05/27 18:14:54 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:41711 ...
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39011 ...
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40461 ...
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2999768 ns
21/05/27 18:14:57 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1562033 ns
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/f7e6ff6ade42008b46ebf8efe2f87707c41556a5534c44deaf09c620ae930aca/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 18:14:57 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 18:14:57 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6c2c4adefc49e2a365ebc407b3cb92cea8489366a8e13e13bac261ad28406915. Logs:
2021/05/27 18:14:57 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622139291205489057"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622139291205489057-jenkins-0527181453-e43af5b2"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39011"} artifact_endpoint:{url:"localhost:40735"} control_endpoint:{url:"localhost:41711"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/f7e6ff6ade42008b46ebf8efe2f87707c41556a5534c44deaf09c620ae930aca/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 18:14:57 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34465
2021/05/27 18:14:57 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/27 18:14:57 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 18:14:57 control response channel closed
21/05/27 18:15:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef finished.
2021/05/27 18:15:02 Job state: DONE
21/05/27 18:15:02 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
21/05/27 18:15:02 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622139291205489057-jenkins-0527181453-e43af5b2_d4c7eb1f-ca87-479b-8929-e58332a5c3ef
2021/05/27 18:15:02 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (38.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 38.263s
2021/05/27 18:15:03 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.020s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 39s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/vqrzs67hrg2bq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4726
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4726/display/redirect?page=changes>
Changes:
[je.ik] [BEAM-12276] Narrow check for output timestamp of event time timers
------------------------------------------
[...truncated 564.48 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 12:11:24 Prepared job with id: go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91 and staging token: go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91
2021/05/27 12:11:24 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622117484070716378
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.go.
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.null.
21/05/27 12:11:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622117484068346854_2f226b0f-5294-481b-a84b-0e64d1a9fe91.
2021/05/27 12:11:26 Staged binary artifact with token:
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
21/05/27 12:11:26 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
2021/05/27 12:11:26 Submitted job: go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
2021/05/27 12:11:26 Job state: STOPPED
2021/05/27 12:11:26 Job state: STARTING
2021/05/27 12:11:26 Job state: RUNNING
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45 on Spark master local
21/05/27 12:11:26 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 12:11:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45: Pipeline translated successfully. Computing outputs
21/05/27 12:11:27 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45393 ...
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45103 ...
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39611 ...
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2901406 ns
21/05/27 12:11:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1665638 ns
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/8470380370064b97241b917bcd4bada56bd64f833352bd76a96cf8a7064fceee/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 12:11:29 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 12:11:29 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 588290e39b9feba425623539839a06f3488d67ef3d664af4f4262706a1ccb668. Logs:
2021/05/27 12:11:29 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622117484068346854"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622117484068346854-jenkins-0527121126-6489ae1e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45103"} artifact_endpoint:{url:"localhost:45895"} control_endpoint:{url:"localhost:45393"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/8470380370064b97241b917bcd4bada56bd64f833352bd76a96cf8a7064fceee/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 12:11:29 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44243
2021/05/27 12:11:29 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/27 12:11:29 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 12:11:29 control response channel closed
21/05/27 12:11:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45 finished.
2021/05/27 12:11:30 Job state: DONE
21/05/27 12:11:30 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
21/05/27 12:11:30 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622117484068346854-jenkins-0527121126-6489ae1e_b88a0294-e902-4e53-a9c9-6b3a8a5aae45
2021/05/27 12:11:30 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (30.21s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 30.234s
2021/05/27 12:11:32 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 9s
79 actionable tasks: 55 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/3ymbsip2iuqds
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4725
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4725/display/redirect>
Changes:
------------------------------------------
[...truncated 565.04 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 06:14:40 Prepared job with id: go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea and staging token: go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea
2021/05/27 06:14:40 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622096080096767958
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.go.
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.null.
21/05/27 06:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622096080093745755_f9677a65-3617-4a71-8c07-a05f888f2fea.
2021/05/27 06:14:43 Staged binary artifact with token:
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
21/05/27 06:14:43 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
2021/05/27 06:14:43 Submitted job: go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
2021/05/27 06:14:43 Job state: STOPPED
2021/05/27 06:14:43 Job state: STARTING
2021/05/27 06:14:43 Job state: RUNNING
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae on Spark master local
21/05/27 06:14:43 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 06:14:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae: Pipeline translated successfully. Computing outputs
21/05/27 06:14:43 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37627 ...
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34229 ...
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42983 ...
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3522117 ns
21/05/27 06:14:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1902962 ns
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/7fb59c4b154073c19eabd6391454962ce7c2ac446748af7f7a4b6486360181b8/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 06:14:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 06:14:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ce25e6dc2628cb4966dbe8a631439f6a83401d47f38b7941a59a3c7a064ff479. Logs:
2021/05/27 06:14:44 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622096080093745755"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622096080093745755-jenkins-0527061443-f2adbfe6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34229"} artifact_endpoint:{url:"localhost:35213"} control_endpoint:{url:"localhost:37627"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/7fb59c4b154073c19eabd6391454962ce7c2ac446748af7f7a4b6486360181b8/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 06:14:44 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:35215
2021/05/27 06:14:44 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/27 06:14:44 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 06:14:44 control response channel closed
21/05/27 06:14:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae finished.
2021/05/27 06:14:45 Job state: DONE
21/05/27 06:14:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
21/05/27 06:14:45 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622096080093745755-jenkins-0527061443-f2adbfe6_a31833c8-07a7-4968-a686-7b4fb8486bae
2021/05/27 06:14:45 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.57s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.595s
2021/05/27 06:14:47 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.021s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 19s
79 actionable tasks: 53 executed, 26 from cache
Publishing build scan...
https://gradle.com/s/qobi2yab7w3lg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4724
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4724/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12240] Add Java 17 as a valid environment
[emilyye] update release tag for Java UW container to not have beam-prefix
[zyichi] Bump dataflow java legacy container version to beam-master-20210525
[zyichi] Bump python container versions to beam-master-20210526
[noreply] [BEAM-9547] Add additional unliftable grouped aggregations, add describe
[noreply] [BEAM-12024] Add DataFrame example pipeline README (#14886)
[noreply] Merge pull request #14829 from [BEAM-11838] An x-lang wrapper for
[noreply] [BEAM-12074] Define docstrings for all Series and StringMethods
------------------------------------------
[...truncated 566.29 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/27 00:17:02 Prepared job with id: go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1 and staging token: go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1
2021/05/27 00:17:02 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622074622729723323
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.go.
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.null.
21/05/27 00:17:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622074622726624809_090c62e5-f755-4d33-b9de-fa65fe243ad1.
2021/05/27 00:17:06 Staged binary artifact with token:
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
21/05/27 00:17:06 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
2021/05/27 00:17:06 Submitted job: go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
2021/05/27 00:17:06 Job state: STOPPED
2021/05/27 00:17:06 Job state: STARTING
2021/05/27 00:17:06 Job state: RUNNING
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb on Spark master local
21/05/27 00:17:06 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/27 00:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb: Pipeline translated successfully. Computing outputs
21/05/27 00:17:06 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39303 ...
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46153 ...
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46549 ...
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3247548 ns
21/05/27 00:17:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1811480 ns
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e7d213f052690ea783fd8910492d636694fcfb1811f27e7d655fbc7291dee603/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/27 00:17:16 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/27 00:17:16 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6280802e47bbf4152ef4672fd9a71aa82ce89dd753df63a2cf502236a1f98e0b. Logs:
2021/05/27 00:17:16 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622074622726624809"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622074622726624809-jenkins-0527001706-1710cb5e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46153"} artifact_endpoint:{url:"localhost:44745"} control_endpoint:{url:"localhost:39303"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e7d213f052690ea783fd8910492d636694fcfb1811f27e7d655fbc7291dee603/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/27 00:17:16 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:43623
2021/05/27 00:17:16 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/27 00:17:16 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/27 00:17:16 control response channel closed
21/05/27 00:17:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb finished.
2021/05/27 00:17:17 Job state: DONE
21/05/27 00:17:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
21/05/27 00:17:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622074622726624809-jenkins-0527001706-1710cb5e_b98418a2-557b-43d7-827c-01b564256cfb
2021/05/27 00:17:17 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (40.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 40.260s
2021/05/27 00:17:19 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 51s
79 actionable tasks: 62 executed, 17 from cache
Publishing build scan...
https://gradle.com/s/xblath2yd73fg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4723
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4723/display/redirect>
Changes:
------------------------------------------
[...truncated 564.52 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 18:14:09 Prepared job with id: go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae and staging token: go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae
2021/05/26 18:14:09 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622052849798635224
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.go.
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.null.
21/05/26 18:14:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622052849795815379_adc4b0b7-8f67-4a1f-a873-f589ff5273ae.
2021/05/26 18:14:12 Staged binary artifact with token:
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
21/05/26 18:14:12 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
2021/05/26 18:14:12 Submitted job: go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
2021/05/26 18:14:12 Job state: STOPPED
2021/05/26 18:14:12 Job state: STARTING
2021/05/26 18:14:12 Job state: RUNNING
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f on Spark master local
21/05/26 18:14:12 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 18:14:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f: Pipeline translated successfully. Computing outputs
21/05/26 18:14:13 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39913 ...
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33611 ...
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33919 ...
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3500968 ns
21/05/26 18:14:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1817399 ns
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e885e975aaad63eadee37971f87ef45a0ed1daa9929407170bb622cfa446b93d/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 18:14:14 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 18:14:14 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container caeef51a5ab9e13aa68daf181d4d1b9d671466df32cdb5b25b33a9a15e6ee23d. Logs:
2021/05/26 18:14:14 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622052849795815379"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622052849795815379-jenkins-0526181412-1d9edd4a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33611"} artifact_endpoint:{url:"localhost:41671"} control_endpoint:{url:"localhost:39913"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e885e975aaad63eadee37971f87ef45a0ed1daa9929407170bb622cfa446b93d/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 18:14:14 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40505
2021/05/26 18:14:14 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 18:14:14 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 18:14:14 control response channel closed
21/05/26 18:14:15 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f finished.
2021/05/26 18:14:15 Job state: DONE
21/05/26 18:14:15 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
21/05/26 18:14:15 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622052849795815379-jenkins-0526181412-1d9edd4a_b6f8a230-b2cb-417b-9e2b-ccb65b86f00f
2021/05/26 18:14:15 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.55s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.576s
2021/05/26 18:14:16 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.020s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 48s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/e4n4wfoh5l2pu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4722
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4722/display/redirect>
Changes:
------------------------------------------
[...truncated 564.58 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 12:10:49 Prepared job with id: go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c and staging token: go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c
2021/05/26 12:10:49 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622031049688025451
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.go.
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.null.
21/05/26 12:10:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622031049684592910_2f1d21df-e25d-49f5-b28f-1ffa04414b8c.
2021/05/26 12:10:52 Staged binary artifact with token:
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
21/05/26 12:10:52 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
2021/05/26 12:10:52 Submitted job: go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
2021/05/26 12:10:52 Job state: STOPPED
2021/05/26 12:10:52 Job state: STARTING
2021/05/26 12:10:52 Job state: RUNNING
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8 on Spark master local
21/05/26 12:10:52 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 12:10:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8: Pipeline translated successfully. Computing outputs
21/05/26 12:10:53 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:32883 ...
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42805 ...
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36755 ...
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3319676 ns
21/05/26 12:10:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1762913 ns
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/e79d94334f184a5b1f4e0db8be3a774d535c539243140f0ddeb0b83c2e69c683/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 12:10:54 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 12:10:54 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e1127a5cfaae0a7c9ec06cb6b1feee9d678f73758314afbf026f0b883cba706b. Logs:
2021/05/26 12:10:53 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622031049684592910"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622031049684592910-jenkins-0526121052-22fb2554"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42805"} artifact_endpoint:{url:"localhost:41073"} control_endpoint:{url:"localhost:32883"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/e79d94334f184a5b1f4e0db8be3a774d535c539243140f0ddeb0b83c2e69c683/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 12:10:53 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:33913
2021/05/26 12:10:54 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 12:10:54 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 12:10:54 control response channel closed
21/05/26 12:10:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8 finished.
2021/05/26 12:10:54 Job state: DONE
21/05/26 12:10:54 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
21/05/26 12:10:54 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622031049684592910-jenkins-0526121052-22fb2554_604a36cd-b5f7-4d11-bf2a-c1b63e89e8d8
2021/05/26 12:10:54 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.24s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.258s
2021/05/26 12:10:56 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 28s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/tvwfpztqwgtea
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4721
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4721/display/redirect?page=changes>
Changes:
[yoshiki.obata] [BEAM-7372] cleanup py2 codepath
[noreply] [BEAM-12107] Add integration test script for taxiride example (#14882)
[noreply] Add dataflow, tfx to powered by page (#14820)
[noreply] Merge pull request #14765: [BEAM-12405] Reduce tail latency in Storage
------------------------------------------
[...truncated 566.43 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 06:15:59 Prepared job with id: go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8 and staging token: go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8
2021/05/26 06:15:59 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1622009759142978740
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.go.
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.null.
21/05/26 06:16:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1622009759139804762_9bbfeb9e-8df3-4422-98dd-14eca41547c8.
2021/05/26 06:16:02 Staged binary artifact with token:
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
21/05/26 06:16:02 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
2021/05/26 06:16:02 Submitted job: go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
2021/05/26 06:16:02 Job state: STOPPED
2021/05/26 06:16:02 Job state: STARTING
2021/05/26 06:16:02 Job state: RUNNING
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce on Spark master local
21/05/26 06:16:02 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 06:16:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce: Pipeline translated successfully. Computing outputs
21/05/26 06:16:02 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34273 ...
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40191 ...
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:32793 ...
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3414942 ns
21/05/26 06:16:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1857321 ns
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/82e636dc39d28d125f85714f473df91197f34f800d8a34de63d97c19659f82f0/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 06:16:04 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 06:16:04 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container eff2e956f5d1304993a7db527cdef921194359561bd1b3cad78c1436c7ebf5a6. Logs:
2021/05/26 06:16:03 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1622009759139804762"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501622009759139804762-jenkins-0526061602-2eba000"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40191"} artifact_endpoint:{url:"localhost:37403"} control_endpoint:{url:"localhost:34273"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/82e636dc39d28d125f85714f473df91197f34f800d8a34de63d97c19659f82f0/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 06:16:03 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34919
2021/05/26 06:16:03 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 06:16:04 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 06:16:04 control response channel closed
21/05/26 06:16:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce finished.
2021/05/26 06:16:05 Job state: DONE
21/05/26 06:16:05 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
21/05/26 06:16:05 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501622009759139804762-jenkins-0526061602-2eba000_2b9ce507-7c9c-4b3e-b584-9849b10428ce
2021/05/26 06:16:05 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (32.62s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 32.641s
2021/05/26 06:16:06 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.022s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 15m 37s
79 actionable tasks: 56 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/iuemtv23ugtje
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4720
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4720/display/redirect?page=changes>
Changes:
[relax] GroupIntoBatches improvements
[relax] make sure that DataflowRunner batch override for GroupIntoBatches
[noreply] Minor: Fix doc left nav on dev website (#14881)
[zyichi] [BEAM-12395] Add support to enable google cloud profiler for python SDK
[relax] fix DataflowRunnerTest
[noreply] Fix flaky test on slow remote systems by adding `afterIterations` to
------------------------------------------
[...truncated 567.43 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/26 00:22:52 Prepared job with id: go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f and staging token: go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f
2021/05/26 00:22:52 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621988572537355649
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.go.
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.null.
21/05/26 00:22:59 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621988572532376737_9e1828c1-cc34-44d2-bc4b-20c749388a3f.
2021/05/26 00:22:59 Staged binary artifact with token:
21/05/26 00:22:59 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
21/05/26 00:22:59 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
2021/05/26 00:22:59 Submitted job: go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
2021/05/26 00:22:59 Job state: STOPPED
2021/05/26 00:22:59 Job state: STARTING
2021/05/26 00:22:59 Job state: RUNNING
21/05/26 00:22:59 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/26 00:23:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb on Spark master local
21/05/26 00:23:00 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/26 00:23:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb: Pipeline translated successfully. Computing outputs
21/05/26 00:23:01 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34863 ...
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42685 ...
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33965 ...
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 6534702 ns
21/05/26 00:23:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 4091085 ns
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/c52d50c5432d2f88f1f1fc9896bc5215d5abd233c62f9cbde81036057f9ebae1/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/26 00:23:03 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/26 00:23:03 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 76fca25bca8b75423e0faf30b1999f18b965421f90bea91c065a752a0a39bb2c. Logs:
2021/05/26 00:23:02 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621988572532376737"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621988572532376737-jenkins-0526002259-75c7f446"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42685"} artifact_endpoint:{url:"localhost:33745"} control_endpoint:{url:"localhost:34863"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/c52d50c5432d2f88f1f1fc9896bc5215d5abd233c62f9cbde81036057f9ebae1/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/26 00:23:02 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34815
2021/05/26 00:23:03 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/26 00:23:03 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/26 00:23:03 control response channel closed
21/05/26 00:23:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb finished.
2021/05/26 00:23:04 Job state: DONE
21/05/26 00:23:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
21/05/26 00:23:04 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621988572532376737-jenkins-0526002259-75c7f446_66c855b8-3f50-4ef6-bd04-57c1faf55aeb
2021/05/26 00:23:04 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (60.30s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 60.361s
2021/05/26 00:23:09 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.074s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 22m 47s
79 actionable tasks: 69 executed, 10 from cache
Publishing build scan...
https://gradle.com/s/ddfpri3clq7b6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4719
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4719/display/redirect?page=changes>
Changes:
[noreply] Update test_error_message_includes_stage test
[samuelw] [BEAM-12402] Optimize
[noreply] [BEAM-12284] Use the element coder when pickling beam.Create. (#14834)
------------------------------------------
[...truncated 564.51 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/25 18:14:01 Prepared job with id: go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2 and staging token: go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2
2021/05/25 18:14:01 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621966441225354577
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.go.
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.null.
21/05/25 18:14:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621966441222215423_a8d9f9e6-aaef-4965-bdfa-74ff734833d2.
2021/05/25 18:14:04 Staged binary artifact with token:
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
21/05/25 18:14:04 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
2021/05/25 18:14:04 Submitted job: go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
2021/05/25 18:14:04 Job state: STOPPED
2021/05/25 18:14:04 Job state: STARTING
2021/05/25 18:14:04 Job state: RUNNING
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1 on Spark master local
21/05/25 18:14:04 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/25 18:14:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1: Pipeline translated successfully. Computing outputs
21/05/25 18:14:04 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37291 ...
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40033 ...
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36239 ...
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3740834 ns
21/05/25 18:14:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1945557 ns
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/928ef1ad3445a9ce5592ee07dc95461c6e5015c8be749976e9b76ce26d004659/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/25 18:14:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/25 18:14:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container cf7b389edf596769b82f9e93a0c3f0edd4be87c98a697478b4f6fe173ce89d35. Logs:
2021/05/25 18:14:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621966441222215423"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621966441222215423-jenkins-0525181404-ee8c4d02"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40033"} artifact_endpoint:{url:"localhost:41285"} control_endpoint:{url:"localhost:37291"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/928ef1ad3445a9ce5592ee07dc95461c6e5015c8be749976e9b76ce26d004659/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/25 18:14:05 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:46509
2021/05/25 18:14:05 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/25 18:14:05 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/25 18:14:05 control response channel closed
21/05/25 18:14:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1 finished.
2021/05/25 18:14:06 Job state: DONE
21/05/25 18:14:06 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
21/05/25 18:14:06 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621966441222215423-jenkins-0525181404-ee8c4d02_abd6f1b1-b3d1-4123-843f-627996910fa1
2021/05/25 18:14:06 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (28.44s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 28.460s
2021/05/25 18:14:08 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 44s
79 actionable tasks: 57 executed, 22 from cache
Publishing build scan...
https://gradle.com/s/leobk3lh47ipm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4718
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4718/display/redirect?page=changes>
Changes:
[emilyye] change default UW image for java to beam_java#_sdk
------------------------------------------
[...truncated 565.84 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/25 12:12:47 Prepared job with id: go-job-5-1621944767104705631_640b028c-84c0-428b-ad84-4b1b82db4ee3 and staging token: go-job-5-1621944767104705631_640b028c-84c0-428b-ad84-4b1b82db4ee3
2021/05/25 12:12:47 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621944767107778489
21/05/25 12:12:49 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621944767104705631_640b028c-84c0-428b-ad84-4b1b82db4ee3.
21/05/25 12:12:49 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621944767104705631_640b028c-84c0-428b-ad84-4b1b82db4ee3.go.
21/05/25 12:12:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621944767104705631_640b028c-84c0-428b-ad84-4b1b82db4ee3.null.
21/05/25 12:12:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621944767104705631_640b028c-84c0-428b-ad84-4b1b82db4ee3.
2021/05/25 12:12:50 Staged binary artifact with token:
21/05/25 12:12:50 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0
21/05/25 12:12:50 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0
2021/05/25 12:12:50 Submitted job: go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0
2021/05/25 12:12:50 Job state: STOPPED
2021/05/25 12:12:50 Job state: STARTING
2021/05/25 12:12:50 Job state: RUNNING
21/05/25 12:12:50 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/25 12:12:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0 on Spark master local
21/05/25 12:12:50 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/25 12:12:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0: Pipeline translated successfully. Computing outputs
21/05/25 12:12:50 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/25 12:12:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/25 12:12:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/25 12:12:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45951 ...
21/05/25 12:12:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45773 ...
21/05/25 12:12:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/25 12:12:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36881 ...
21/05/25 12:12:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/25 12:12:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3872781 ns
21/05/25 12:12:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2006943 ns
21/05/25 12:12:51 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/d2391b9273181e8960829db6a026974257e0cee5f330c83d39673e15299d93a2/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/25 12:12:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/25 12:12:51 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/25 12:12:51 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 45f55c543735b04a5ed9d41b633f04ddec56cc7d42c5d14e1b93f00c7886edc2. Logs:
2021/05/25 12:12:51 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621944767104705631"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45773"} artifact_endpoint:{url:"localhost:35655"} control_endpoint:{url:"localhost:45951"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/d2391b9273181e8960829db6a026974257e0cee5f330c83d39673e15299d93a2/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/25 12:12:51 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:43637
2021/05/25 12:12:51 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/25 12:12:51 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/25 12:12:51 control response channel closed
21/05/25 12:12:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0 finished.
2021/05/25 12:12:52 Job state: DONE
21/05/25 12:12:52 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0
21/05/25 12:12:52 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621944767104705631-jenkins-0525121250-6c7b8a1c_8f61414b-6757-48a7-bbbc-df3d38b928b0
2021/05/25 12:12:52 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (29.63s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 29.651s
2021/05/25 12:12:54 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 32s
79 actionable tasks: 59 executed, 20 from cache
Publishing build scan...
https://gradle.com/s/lxihxjj434y2u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4717
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4717/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12394] Include UDF provider jar in Hadoop variant test.
[Kyle Weaver] [BEAM-11738] Fail JavaUdfLoaderTest instead of ignoring if system
------------------------------------------
[...truncated 564.47 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/25 06:15:00 Prepared job with id: go-job-5-1621923300590914070_aa2b6c2b-8875-401c-9d3b-f1cc01a7cd86 and staging token: go-job-5-1621923300590914070_aa2b6c2b-8875-401c-9d3b-f1cc01a7cd86
2021/05/25 06:15:00 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621923300593985993
21/05/25 06:15:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621923300590914070_aa2b6c2b-8875-401c-9d3b-f1cc01a7cd86.
21/05/25 06:15:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621923300590914070_aa2b6c2b-8875-401c-9d3b-f1cc01a7cd86.go.
21/05/25 06:15:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621923300590914070_aa2b6c2b-8875-401c-9d3b-f1cc01a7cd86.null.
21/05/25 06:15:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621923300590914070_aa2b6c2b-8875-401c-9d3b-f1cc01a7cd86.
2021/05/25 06:15:03 Staged binary artifact with token:
21/05/25 06:15:03 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312
21/05/25 06:15:03 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312
2021/05/25 06:15:03 Submitted job: go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312
2021/05/25 06:15:03 Job state: STOPPED
2021/05/25 06:15:03 Job state: STARTING
2021/05/25 06:15:03 Job state: RUNNING
21/05/25 06:15:03 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/25 06:15:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312 on Spark master local
21/05/25 06:15:03 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/25 06:15:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312: Pipeline translated successfully. Computing outputs
21/05/25 06:15:04 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/25 06:15:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/25 06:15:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/25 06:15:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45455 ...
21/05/25 06:15:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35923 ...
21/05/25 06:15:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/25 06:15:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40147 ...
21/05/25 06:15:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/25 06:15:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3823873 ns
21/05/25 06:15:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2029834 ns
21/05/25 06:15:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/364f73e83bedda1ebc7eed568b0063652939f0a564926dd675634eafb0576947/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/25 06:15:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/25 06:15:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/25 06:15:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container a0efbb95840de62ea76c5b0e4822cc43fca324ed65682875bf8f9688185fdb30. Logs:
2021/05/25 06:15:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621923300590914070"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621923300590914070-jenkins-0525061503-f6883a98"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35923"} artifact_endpoint:{url:"localhost:39169"} control_endpoint:{url:"localhost:45455"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/364f73e83bedda1ebc7eed568b0063652939f0a564926dd675634eafb0576947/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/25 06:15:05 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:37157
2021/05/25 06:15:05 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/25 06:15:05 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/25 06:15:05 control response channel closed
21/05/25 06:15:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312 finished.
2021/05/25 06:15:06 Job state: DONE
21/05/25 06:15:06 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312
21/05/25 06:15:06 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621923300590914070-jenkins-0525061503-f6883a98_f1da8f6f-5a44-4b1c-838d-30634cf94312
2021/05/25 06:15:06 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (28.44s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 28.462s
2021/05/25 06:15:08 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.021s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 43s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/rzedntb7edndc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4716
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4716/display/redirect?page=changes>
Changes:
[heejong] [BEAM-12390] clickhouse test fails when test resource can't be read by
[Ismaël Mejía] [BEAM-12241] Update vendored bytebuddy to version 1.11.0
[noreply] [BEAM-10144] update pipeline options snippets (#14738)
[Kyle Weaver] Update Dataflow Python containers.
------------------------------------------
[...truncated 567.82 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/25 00:24:04 Prepared job with id: go-job-5-1621902244171145714_14c5691d-b795-41c4-9164-7594b8ddf945 and staging token: go-job-5-1621902244171145714_14c5691d-b795-41c4-9164-7594b8ddf945
2021/05/25 00:24:04 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621902244175447405
21/05/25 00:24:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621902244171145714_14c5691d-b795-41c4-9164-7594b8ddf945.
21/05/25 00:24:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621902244171145714_14c5691d-b795-41c4-9164-7594b8ddf945.go.
21/05/25 00:24:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621902244171145714_14c5691d-b795-41c4-9164-7594b8ddf945.null.
21/05/25 00:24:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621902244171145714_14c5691d-b795-41c4-9164-7594b8ddf945.
2021/05/25 00:24:11 Staged binary artifact with token:
21/05/25 00:24:11 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b
21/05/25 00:24:11 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b
2021/05/25 00:24:11 Submitted job: go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b
21/05/25 00:24:11 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
2021/05/25 00:24:11 Job state: STOPPED
2021/05/25 00:24:11 Job state: STARTING
2021/05/25 00:24:11 Job state: RUNNING
21/05/25 00:24:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b on Spark master local
21/05/25 00:24:11 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/25 00:24:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b: Pipeline translated successfully. Computing outputs
21/05/25 00:24:12 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/25 00:24:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/25 00:24:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/25 00:24:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37771 ...
21/05/25 00:24:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33093 ...
21/05/25 00:24:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/25 00:24:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45675 ...
21/05/25 00:24:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/25 00:24:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 9647961 ns
21/05/25 00:24:16 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2790593 ns
21/05/25 00:24:16 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/2239afe0a18d767f1e3e655e482c7f78a5f0362b9e30e32467ec4f454d778b40/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/25 00:24:16 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/25 00:24:16 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/25 00:24:16 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 7a4764547e5ff8dde0d8ab914b226ec8f86b6c09caf2a488df51eb9d07135665. Logs:
2021/05/25 00:24:15 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621902244171145714"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33093"} artifact_endpoint:{url:"localhost:36741"} control_endpoint:{url:"localhost:37771"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/2239afe0a18d767f1e3e655e482c7f78a5f0362b9e30e32467ec4f454d778b40/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/25 00:24:15 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:37879
2021/05/25 00:24:15 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/25 00:24:15 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/25 00:24:16 control response channel closed
21/05/25 00:24:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b finished.
2021/05/25 00:24:18 Job state: DONE
21/05/25 00:24:18 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b
21/05/25 00:24:18 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621902244171145714-jenkins-0525002411-4e0bc6b5_5396a207-2000-4eb9-8347-d39886a16e1b
2021/05/25 00:24:18 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (79.11s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 79.181s
2021/05/25 00:24:22 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.097s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 24m 0s
79 actionable tasks: 72 executed, 7 from cache
Publishing build scan...
https://gradle.com/s/z24csor2ujgfk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4715
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4715/display/redirect?page=changes>
Changes:
[samuelw] [BEAM-7717] Fix PubsubGrpcClient timestamp parsing
------------------------------------------
[...truncated 565.22 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/24 18:15:40 Prepared job with id: go-job-5-1621880140580789561_12c748d1-f653-4344-b4d0-64508ca86237 and staging token: go-job-5-1621880140580789561_12c748d1-f653-4344-b4d0-64508ca86237
2021/05/24 18:15:40 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621880140584302116
21/05/24 18:15:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621880140580789561_12c748d1-f653-4344-b4d0-64508ca86237.
21/05/24 18:15:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621880140580789561_12c748d1-f653-4344-b4d0-64508ca86237.go.
21/05/24 18:15:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621880140580789561_12c748d1-f653-4344-b4d0-64508ca86237.null.
21/05/24 18:15:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621880140580789561_12c748d1-f653-4344-b4d0-64508ca86237.
2021/05/24 18:15:44 Staged binary artifact with token:
21/05/24 18:15:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4
21/05/24 18:15:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4
2021/05/24 18:15:44 Submitted job: go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4
2021/05/24 18:15:44 Job state: STOPPED
2021/05/24 18:15:44 Job state: STARTING
2021/05/24 18:15:44 Job state: RUNNING
21/05/24 18:15:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/24 18:15:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4 on Spark master local
21/05/24 18:15:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/24 18:15:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4: Pipeline translated successfully. Computing outputs
21/05/24 18:15:44 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/24 18:15:45 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/24 18:15:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/24 18:15:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37841 ...
21/05/24 18:15:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35213 ...
21/05/24 18:15:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/24 18:15:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35195 ...
21/05/24 18:15:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/24 18:15:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3892929 ns
21/05/24 18:15:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2161621 ns
21/05/24 18:15:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/c99af5ca96156ee881ae918060e819d1292534b688815cf366d5c19e64c4ce67/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/24 18:15:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/24 18:15:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/24 18:15:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 5b39a8049d2aa16f594cc6a4c3dfc2856869a97c5d8f5a7656dcab24c26b5500. Logs:
2021/05/24 18:15:45 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621880140580789561"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621880140580789561-jenkins-0524181544-e688e390"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35213"} artifact_endpoint:{url:"localhost:36037"} control_endpoint:{url:"localhost:37841"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/c99af5ca96156ee881ae918060e819d1292534b688815cf366d5c19e64c4ce67/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/24 18:15:45 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44711
2021/05/24 18:15:45 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/24 18:15:45 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/24 18:15:46 control response channel closed
21/05/24 18:15:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4 finished.
2021/05/24 18:15:46 Job state: DONE
21/05/24 18:15:46 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4
21/05/24 18:15:46 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621880140580789561-jenkins-0524181544-e688e390_37407701-9b0f-48f7-9649-1cd8590205e4
2021/05/24 18:15:46 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (38.74s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 38.772s
2021/05/24 18:15:48 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 15m 23s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/z524qjjmgh2iu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4714
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4714/display/redirect>
Changes:
------------------------------------------
[...truncated 563.50 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/24 12:12:37 Prepared job with id: go-job-5-1621858357552560540_1cd4948b-a6a5-45e5-88e7-1c6bce2307dc and staging token: go-job-5-1621858357552560540_1cd4948b-a6a5-45e5-88e7-1c6bce2307dc
2021/05/24 12:12:37 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621858357555980866
21/05/24 12:12:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621858357552560540_1cd4948b-a6a5-45e5-88e7-1c6bce2307dc.
21/05/24 12:12:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621858357552560540_1cd4948b-a6a5-45e5-88e7-1c6bce2307dc.go.
21/05/24 12:12:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621858357552560540_1cd4948b-a6a5-45e5-88e7-1c6bce2307dc.null.
21/05/24 12:12:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621858357552560540_1cd4948b-a6a5-45e5-88e7-1c6bce2307dc.
2021/05/24 12:12:40 Staged binary artifact with token:
21/05/24 12:12:40 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4
21/05/24 12:12:40 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4
2021/05/24 12:12:40 Submitted job: go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4
2021/05/24 12:12:40 Job state: STOPPED
2021/05/24 12:12:40 Job state: STARTING
2021/05/24 12:12:40 Job state: RUNNING
21/05/24 12:12:40 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/24 12:12:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4 on Spark master local
21/05/24 12:12:40 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/24 12:12:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4: Pipeline translated successfully. Computing outputs
21/05/24 12:12:40 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/24 12:12:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/24 12:12:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/24 12:12:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36729 ...
21/05/24 12:12:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44475 ...
21/05/24 12:12:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/24 12:12:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:39673 ...
21/05/24 12:12:42 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/24 12:12:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3681538 ns
21/05/24 12:12:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1938722 ns
21/05/24 12:12:42 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/9029d7a289043ff2d7ad11ac82adbea78410157f1fa756a9ebf31f841d605613/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/24 12:12:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/24 12:12:42 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/24 12:12:42 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b864f742360bb8929e140d1b216e7d0870d6cbdfded650df789dca636b77cde5. Logs:
2021/05/24 12:12:41 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621858357552560540"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621858357552560540-jenkins-0524121240-6b93cf80"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44475"} artifact_endpoint:{url:"localhost:34097"} control_endpoint:{url:"localhost:36729"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/9029d7a289043ff2d7ad11ac82adbea78410157f1fa756a9ebf31f841d605613/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/24 12:12:41 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:42813
2021/05/24 12:12:41 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/24 12:12:42 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/24 12:12:42 control response channel closed
21/05/24 12:12:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4 finished.
2021/05/24 12:12:42 Job state: DONE
21/05/24 12:12:42 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4
21/05/24 12:12:42 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621858357552560540-jenkins-0524121240-6b93cf80_af780cf9-9ae5-41d4-9d7a-1e7e3118fae4
2021/05/24 12:12:42 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (28.76s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 28.786s
2021/05/24 12:12:44 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 22s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/faqozrm6i5kr6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4713
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4713/display/redirect>
Changes:
------------------------------------------
[...truncated 564.71 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/24 06:13:49 Prepared job with id: go-job-5-1621836829296186667_83e21847-ccc7-4fb9-9442-903b74e71daf and staging token: go-job-5-1621836829296186667_83e21847-ccc7-4fb9-9442-903b74e71daf
2021/05/24 06:13:49 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621836829298889649
21/05/24 06:13:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621836829296186667_83e21847-ccc7-4fb9-9442-903b74e71daf.
21/05/24 06:13:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621836829296186667_83e21847-ccc7-4fb9-9442-903b74e71daf.go.
21/05/24 06:13:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621836829296186667_83e21847-ccc7-4fb9-9442-903b74e71daf.null.
21/05/24 06:13:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621836829296186667_83e21847-ccc7-4fb9-9442-903b74e71daf.
2021/05/24 06:13:52 Staged binary artifact with token:
21/05/24 06:13:52 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844
21/05/24 06:13:52 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844
2021/05/24 06:13:52 Submitted job: go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844
2021/05/24 06:13:52 Job state: STOPPED
2021/05/24 06:13:52 Job state: STARTING
2021/05/24 06:13:52 Job state: RUNNING
21/05/24 06:13:52 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/24 06:13:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844 on Spark master local
21/05/24 06:13:52 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/24 06:13:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844: Pipeline translated successfully. Computing outputs
21/05/24 06:13:52 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/24 06:13:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/24 06:13:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/24 06:13:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38393 ...
21/05/24 06:13:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44095 ...
21/05/24 06:13:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/24 06:13:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43329 ...
21/05/24 06:13:53 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/24 06:13:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3832274 ns
21/05/24 06:13:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2107655 ns
21/05/24 06:13:53 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/5a4dbe7789458c090fa5b1c81f5bd84d625ad2459083782ea16600d75fc7a319/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/24 06:13:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/24 06:13:53 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/24 06:13:54 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9566255bb1d324e03ac2d7b0b52e058daa772b05d9c61d3896842be8f3b84871. Logs:
2021/05/24 06:13:53 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621836829296186667"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621836829296186667-jenkins-0524061352-a2e8b555"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44095"} artifact_endpoint:{url:"localhost:37407"} control_endpoint:{url:"localhost:38393"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/5a4dbe7789458c090fa5b1c81f5bd84d625ad2459083782ea16600d75fc7a319/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/24 06:13:53 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34865
2021/05/24 06:13:53 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/24 06:13:53 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/24 06:13:53 control response channel closed
21/05/24 06:13:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844 finished.
2021/05/24 06:13:54 Job state: DONE
21/05/24 06:13:54 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844
21/05/24 06:13:54 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621836829296186667-jenkins-0524061352-a2e8b555_5cdc9620-d3ac-4ceb-8539-82756ee9c844
2021/05/24 06:13:54 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.62s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.640s
2021/05/24 06:13:56 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 35s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/kn7rujrc6sj6w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4712
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4712/display/redirect>
Changes:
------------------------------------------
[...truncated 564.35 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/24 00:14:41 Prepared job with id: go-job-5-1621815281895581892_d507babb-c0b8-4c14-8c27-06093f460c81 and staging token: go-job-5-1621815281895581892_d507babb-c0b8-4c14-8c27-06093f460c81
2021/05/24 00:14:41 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621815281898638158
21/05/24 00:14:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621815281895581892_d507babb-c0b8-4c14-8c27-06093f460c81.
21/05/24 00:14:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621815281895581892_d507babb-c0b8-4c14-8c27-06093f460c81.go.
21/05/24 00:14:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621815281895581892_d507babb-c0b8-4c14-8c27-06093f460c81.null.
21/05/24 00:14:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621815281895581892_d507babb-c0b8-4c14-8c27-06093f460c81.
2021/05/24 00:14:44 Staged binary artifact with token:
21/05/24 00:14:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e
21/05/24 00:14:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e
2021/05/24 00:14:44 Submitted job: go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e
2021/05/24 00:14:44 Job state: STOPPED
2021/05/24 00:14:44 Job state: STARTING
2021/05/24 00:14:44 Job state: RUNNING
21/05/24 00:14:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/24 00:14:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e on Spark master local
21/05/24 00:14:44 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/24 00:14:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e: Pipeline translated successfully. Computing outputs
21/05/24 00:14:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/24 00:14:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/24 00:14:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/24 00:14:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40875 ...
21/05/24 00:14:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45937 ...
21/05/24 00:14:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/24 00:14:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42855 ...
21/05/24 00:14:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/24 00:14:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3015014 ns
21/05/24 00:14:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1667431 ns
21/05/24 00:14:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/4d4583b1bcd83380f2cee503a1f4d200f763ca7110582147e2a95b4d20ffc197/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/24 00:14:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/24 00:14:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/24 00:14:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6f1e224bb829149d151c9d1763001a6c1a92f4f87ff0714a8d93b83ab727b745. Logs:
2021/05/24 00:14:46 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621815281895581892"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621815281895581892-jenkins-0524001444-a0c4cd11"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45937"} artifact_endpoint:{url:"localhost:46709"} control_endpoint:{url:"localhost:40875"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/4d4583b1bcd83380f2cee503a1f4d200f763ca7110582147e2a95b4d20ffc197/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/24 00:14:46 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:36171
2021/05/24 00:14:46 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/24 00:14:46 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/24 00:14:46 control response channel closed
21/05/24 00:14:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e finished.
2021/05/24 00:14:47 Job state: DONE
21/05/24 00:14:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e
21/05/24 00:14:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621815281895581892-jenkins-0524001444-a0c4cd11_1d476513-147b-45b7-b757-f39180721f3e
2021/05/24 00:14:47 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (26.41s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 26.436s
2021/05/24 00:14:48 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 20s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/mlht73zo4wqay
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4711
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4711/display/redirect>
Changes:
------------------------------------------
[...truncated 564.42 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/23 18:15:19 Prepared job with id: go-job-5-1621793719940014084_62077546-6919-4e7d-9257-5294b6bf395c and staging token: go-job-5-1621793719940014084_62077546-6919-4e7d-9257-5294b6bf395c
2021/05/23 18:15:19 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621793719943418133
21/05/23 18:15:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621793719940014084_62077546-6919-4e7d-9257-5294b6bf395c.
21/05/23 18:15:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621793719940014084_62077546-6919-4e7d-9257-5294b6bf395c.go.
21/05/23 18:15:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621793719940014084_62077546-6919-4e7d-9257-5294b6bf395c.null.
21/05/23 18:15:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621793719940014084_62077546-6919-4e7d-9257-5294b6bf395c.
2021/05/23 18:15:23 Staged binary artifact with token:
21/05/23 18:15:23 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612
21/05/23 18:15:23 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612
2021/05/23 18:15:23 Submitted job: go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612
2021/05/23 18:15:23 Job state: STOPPED
2021/05/23 18:15:23 Job state: STARTING
2021/05/23 18:15:23 Job state: RUNNING
21/05/23 18:15:23 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/23 18:15:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612 on Spark master local
21/05/23 18:15:23 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/23 18:15:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612: Pipeline translated successfully. Computing outputs
21/05/23 18:15:23 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/23 18:15:25 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/23 18:15:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/23 18:15:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46747 ...
21/05/23 18:15:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:37259 ...
21/05/23 18:15:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/23 18:15:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:33419 ...
21/05/23 18:15:25 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/23 18:15:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 4096204 ns
21/05/23 18:15:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2327260 ns
21/05/23 18:15:25 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/154165c6881dcc8aa8486967c6156b4af264b27d8661138467c8285c5d7fccbf/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/23 18:15:25 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/23 18:15:25 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/23 18:15:25 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container bfb5b94a05720ec9c9c3975991821458866fa69bd80cdaaf6a9f333304b435ed. Logs:
2021/05/23 18:15:24 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621793719940014084"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621793719940014084-jenkins-0523181523-c45fe404"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37259"} artifact_endpoint:{url:"localhost:39603"} control_endpoint:{url:"localhost:46747"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/154165c6881dcc8aa8486967c6156b4af264b27d8661138467c8285c5d7fccbf/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/23 18:15:24 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:33423
2021/05/23 18:15:24 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/23 18:15:24 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/23 18:15:25 control response channel closed
21/05/23 18:15:25 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612 finished.
2021/05/23 18:15:25 Job state: DONE
21/05/23 18:15:25 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612
21/05/23 18:15:25 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621793719940014084-jenkins-0523181523-c45fe404_35135522-c2de-4489-a130-2a81b1122612
2021/05/23 18:15:25 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (30.93s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 30.954s
2021/05/23 18:15:27 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.020s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 15m 1s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/2umyejhzknxrm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4710
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4710/display/redirect>
Changes:
------------------------------------------
[...truncated 564.29 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/23 12:10:42 Prepared job with id: go-job-5-1621771842179321053_54363c32-06d7-462b-a66a-3cc2bc37251c and staging token: go-job-5-1621771842179321053_54363c32-06d7-462b-a66a-3cc2bc37251c
2021/05/23 12:10:42 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621771842182218053
21/05/23 12:10:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621771842179321053_54363c32-06d7-462b-a66a-3cc2bc37251c.
21/05/23 12:10:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621771842179321053_54363c32-06d7-462b-a66a-3cc2bc37251c.go.
21/05/23 12:10:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621771842179321053_54363c32-06d7-462b-a66a-3cc2bc37251c.null.
21/05/23 12:10:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621771842179321053_54363c32-06d7-462b-a66a-3cc2bc37251c.
2021/05/23 12:10:44 Staged binary artifact with token:
21/05/23 12:10:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624
21/05/23 12:10:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624
2021/05/23 12:10:44 Submitted job: go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624
2021/05/23 12:10:44 Job state: STOPPED
2021/05/23 12:10:44 Job state: STARTING
2021/05/23 12:10:44 Job state: RUNNING
21/05/23 12:10:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/23 12:10:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624 on Spark master local
21/05/23 12:10:45 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/23 12:10:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624: Pipeline translated successfully. Computing outputs
21/05/23 12:10:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/23 12:10:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/23 12:10:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/23 12:10:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38063 ...
21/05/23 12:10:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:36391 ...
21/05/23 12:10:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/23 12:10:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42097 ...
21/05/23 12:10:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/23 12:10:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3982787 ns
21/05/23 12:10:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2367713 ns
21/05/23 12:10:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/0e17bddb30d3fe3c591c100e4bb569d99c8a2bf31df458b5e1c0a27f17970fa7/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/23 12:10:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/23 12:10:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/23 12:10:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ea09acbaf9a39405700ec8b64f56de1127f8124629a273e0a327eb034968b494. Logs:
2021/05/23 12:10:46 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621771842179321053"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621771842179321053-jenkins-0523121044-d612e8b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36391"} artifact_endpoint:{url:"localhost:38737"} control_endpoint:{url:"localhost:38063"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/0e17bddb30d3fe3c591c100e4bb569d99c8a2bf31df458b5e1c0a27f17970fa7/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/23 12:10:46 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:40961
2021/05/23 12:10:46 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/23 12:10:46 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/23 12:10:46 control response channel closed
21/05/23 12:10:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624 finished.
2021/05/23 12:10:47 Job state: DONE
21/05/23 12:10:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624
21/05/23 12:10:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621771842179321053-jenkins-0523121044-d612e8b_b4d22b73-4397-4096-8f99-196526095624
2021/05/23 12:10:47 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (25.86s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 25.881s
2021/05/23 12:10:48 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 25s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/2jtamawwzjcb4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4709
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4709/display/redirect>
Changes:
------------------------------------------
[...truncated 564.40 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/23 06:14:00 Prepared job with id: go-job-5-1621750440443421996_0de25371-23e9-4cc6-9d43-4a0418d3cd6a and staging token: go-job-5-1621750440443421996_0de25371-23e9-4cc6-9d43-4a0418d3cd6a
2021/05/23 06:14:00 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621750440446271577
21/05/23 06:14:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621750440443421996_0de25371-23e9-4cc6-9d43-4a0418d3cd6a.
21/05/23 06:14:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621750440443421996_0de25371-23e9-4cc6-9d43-4a0418d3cd6a.go.
21/05/23 06:14:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621750440443421996_0de25371-23e9-4cc6-9d43-4a0418d3cd6a.null.
21/05/23 06:14:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621750440443421996_0de25371-23e9-4cc6-9d43-4a0418d3cd6a.
2021/05/23 06:14:03 Staged binary artifact with token:
21/05/23 06:14:03 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33
21/05/23 06:14:03 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33
2021/05/23 06:14:03 Submitted job: go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33
2021/05/23 06:14:03 Job state: STOPPED
2021/05/23 06:14:03 Job state: STARTING
2021/05/23 06:14:03 Job state: RUNNING
21/05/23 06:14:03 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/23 06:14:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33 on Spark master local
21/05/23 06:14:03 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/23 06:14:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33: Pipeline translated successfully. Computing outputs
21/05/23 06:14:03 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/23 06:14:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/23 06:14:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/23 06:14:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46865 ...
21/05/23 06:14:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42687 ...
21/05/23 06:14:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/23 06:14:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:34549 ...
21/05/23 06:14:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/23 06:14:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2979834 ns
21/05/23 06:14:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1639411 ns
21/05/23 06:14:04 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/0e8c266505b3af971f8fd956c5e7760637bcd1201e787b793a01eb5c731a1869/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/23 06:14:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/23 06:14:04 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/23 06:14:04 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container a0af854ddf61d1a28817dbde432712a839210c68d1b6bcfa97ecb3059ac12760. Logs:
2021/05/23 06:14:04 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621750440443421996"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621750440443421996-jenkins-0523061403-e557d273"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42687"} artifact_endpoint:{url:"localhost:37677"} control_endpoint:{url:"localhost:46865"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/0e8c266505b3af971f8fd956c5e7760637bcd1201e787b793a01eb5c731a1869/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/23 06:14:04 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:33147
2021/05/23 06:14:04 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/23 06:14:04 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
2021/05/23 06:14:04 control response channel closed
Worker exited successfully!
21/05/23 06:14:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33 finished.
2021/05/23 06:14:05 Job state: DONE
21/05/23 06:14:05 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33
21/05/23 06:14:05 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621750440443421996-jenkins-0523061403-e557d273_1c5e450a-ce51-4c12-ab14-1917f5eaed33
2021/05/23 06:14:05 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.53s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.553s
2021/05/23 06:14:07 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 42s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/mj3raztitbwy2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4708
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4708/display/redirect>
Changes:
------------------------------------------
[...truncated 564.63 KB...]
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/23 00:14:04 Prepared job with id: go-job-5-1621728844858371768_c3d99300-1b21-461c-9552-6402a9f43694 and staging token: go-job-5-1621728844858371768_c3d99300-1b21-461c-9552-6402a9f43694
2021/05/23 00:14:04 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621728844861116606
21/05/23 00:14:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621728844858371768_c3d99300-1b21-461c-9552-6402a9f43694.
21/05/23 00:14:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621728844858371768_c3d99300-1b21-461c-9552-6402a9f43694.go.
21/05/23 00:14:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621728844858371768_c3d99300-1b21-461c-9552-6402a9f43694.null.
21/05/23 00:14:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621728844858371768_c3d99300-1b21-461c-9552-6402a9f43694.
2021/05/23 00:14:07 Staged binary artifact with token:
21/05/23 00:14:07 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157
21/05/23 00:14:07 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157
2021/05/23 00:14:07 Submitted job: go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157
2021/05/23 00:14:07 Job state: STOPPED
2021/05/23 00:14:07 Job state: STARTING
2021/05/23 00:14:07 Job state: RUNNING
21/05/23 00:14:07 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/23 00:14:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157 on Spark master local
21/05/23 00:14:07 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/23 00:14:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157: Pipeline translated successfully. Computing outputs
21/05/23 00:14:08 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/23 00:14:09 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/23 00:14:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/23 00:14:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:40151 ...
21/05/23 00:14:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:43043 ...
21/05/23 00:14:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/23 00:14:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:41407 ...
21/05/23 00:14:09 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/23 00:14:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 2898338 ns
21/05/23 00:14:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1670533 ns
21/05/23 00:14:09 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/dee8357d6481985308173ed629df18a75f4d0db3baaffddd76aec7347ee12191/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/23 00:14:09 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/23 00:14:09 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/23 00:14:09 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container db2a741d2deab007eb9c790e6096db2101b741a6a2c065163fd759fa8052a97a. Logs:
2021/05/23 00:14:08 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621728844858371768"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621728844858371768-jenkins-0523001407-81ee2512"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43043"} artifact_endpoint:{url:"localhost:40909"} control_endpoint:{url:"localhost:40151"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/dee8357d6481985308173ed629df18a75f4d0db3baaffddd76aec7347ee12191/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/23 00:14:08 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:41511
2021/05/23 00:14:09 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/23 00:14:09 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/23 00:14:09 control response channel closed
21/05/23 00:14:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157 finished.
2021/05/23 00:14:10 Job state: DONE
21/05/23 00:14:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157
21/05/23 00:14:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621728844858371768-jenkins-0523001407-81ee2512_4ee681c8-f28d-42c9-9459-56403ecdd157
2021/05/23 00:14:10 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (26.36s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 26.385s
2021/05/23 00:14:11 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 49s
79 actionable tasks: 52 executed, 27 from cache
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/kmw36zdfgcchc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4707
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4707/display/redirect>
Changes:
------------------------------------------
[...truncated 564.99 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/22 18:14:33 Prepared job with id: go-job-5-1621707273086282760_20d2395c-995f-4771-a580-b97ec7ce4bce and staging token: go-job-5-1621707273086282760_20d2395c-995f-4771-a580-b97ec7ce4bce
2021/05/22 18:14:33 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621707273089245938
21/05/22 18:14:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621707273086282760_20d2395c-995f-4771-a580-b97ec7ce4bce.
21/05/22 18:14:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621707273086282760_20d2395c-995f-4771-a580-b97ec7ce4bce.go.
21/05/22 18:14:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621707273086282760_20d2395c-995f-4771-a580-b97ec7ce4bce.null.
21/05/22 18:14:35 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621707273086282760_20d2395c-995f-4771-a580-b97ec7ce4bce.
2021/05/22 18:14:35 Staged binary artifact with token:
21/05/22 18:14:35 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428
21/05/22 18:14:35 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428
2021/05/22 18:14:35 Submitted job: go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428
2021/05/22 18:14:35 Job state: STOPPED
2021/05/22 18:14:35 Job state: STARTING
2021/05/22 18:14:35 Job state: RUNNING
21/05/22 18:14:35 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/22 18:14:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428 on Spark master local
21/05/22 18:14:36 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/22 18:14:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428: Pipeline translated successfully. Computing outputs
21/05/22 18:14:36 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/22 18:14:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/22 18:14:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/22 18:14:37 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:45973 ...
21/05/22 18:14:37 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35587 ...
21/05/22 18:14:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/22 18:14:37 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:46451 ...
21/05/22 18:14:37 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/22 18:14:37 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3294532 ns
21/05/22 18:14:37 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1958129 ns
21/05/22 18:14:37 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/300785d67b9f9874aa50272cf3a206650bcf76518a19e49e2b894333df7b328b/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/22 18:14:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/22 18:14:37 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/22 18:14:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 028353d3b593f6b809f67714a79a4edcfcb734d6ee8f088354979be645912cf0. Logs:
2021/05/22 18:14:37 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621707273086282760"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621707273086282760-jenkins-0522181435-7047b560"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35587"} artifact_endpoint:{url:"localhost:42949"} control_endpoint:{url:"localhost:45973"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/300785d67b9f9874aa50272cf3a206650bcf76518a19e49e2b894333df7b328b/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/22 18:14:37 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:42945
2021/05/22 18:14:37 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/22 18:14:37 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/22 18:14:37 control response channel closed
21/05/22 18:14:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428 finished.
2021/05/22 18:14:38 Job state: DONE
21/05/22 18:14:38 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428
21/05/22 18:14:38 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621707273086282760-jenkins-0522181435-7047b560_dc832e1f-3767-47b6-a383-edbdf574e428
2021/05/22 18:14:38 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (27.54s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 27.585s
2021/05/22 18:14:40 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.019s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 14m 14s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/jm4bsaq5zk6wy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4706
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4706/display/redirect>
Changes:
------------------------------------------
[...truncated 564.69 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/22 12:11:01 Prepared job with id: go-job-5-1621685461864309493_3e1ec27b-f88f-4bdc-bd51-1268773d7045 and staging token: go-job-5-1621685461864309493_3e1ec27b-f88f-4bdc-bd51-1268773d7045
2021/05/22 12:11:01 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621685461867240966
21/05/22 12:11:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621685461864309493_3e1ec27b-f88f-4bdc-bd51-1268773d7045.
21/05/22 12:11:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621685461864309493_3e1ec27b-f88f-4bdc-bd51-1268773d7045.go.
21/05/22 12:11:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621685461864309493_3e1ec27b-f88f-4bdc-bd51-1268773d7045.null.
21/05/22 12:11:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621685461864309493_3e1ec27b-f88f-4bdc-bd51-1268773d7045.
2021/05/22 12:11:04 Staged binary artifact with token:
21/05/22 12:11:04 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa
21/05/22 12:11:04 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa
2021/05/22 12:11:04 Submitted job: go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa
2021/05/22 12:11:04 Job state: STOPPED
2021/05/22 12:11:04 Job state: STARTING
2021/05/22 12:11:04 Job state: RUNNING
21/05/22 12:11:04 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/22 12:11:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa on Spark master local
21/05/22 12:11:04 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/22 12:11:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa: Pipeline translated successfully. Computing outputs
21/05/22 12:11:05 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/22 12:11:06 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/22 12:11:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/22 12:11:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35717 ...
21/05/22 12:11:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:38937 ...
21/05/22 12:11:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/22 12:11:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:42615 ...
21/05/22 12:11:06 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/22 12:11:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 3566371 ns
21/05/22 12:11:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 1910656 ns
21/05/22 12:11:06 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/6a1290c6847028e62aeec1cbfdd846b3e60e5b285b39c764a6e9294dcc32bea5/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/22 12:11:06 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/22 12:11:06 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/22 12:11:06 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 56fdcb94203eaa603792a8161ce6965630050552efc3e6bf533fe6740dd6e3d6. Logs:
2021/05/22 12:11:06 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621685461864309493"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621685461864309493-jenkins-0522121104-ac07f174"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38937"} artifact_endpoint:{url:"localhost:39081"} control_endpoint:{url:"localhost:35717"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/6a1290c6847028e62aeec1cbfdd846b3e60e5b285b39c764a6e9294dcc32bea5/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/22 12:11:06 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:34771
2021/05/22 12:11:06 Downloaded: /tmp/staged/worker (sha256: 220720d28a64eab9bb29755f675c15bd28cfd2ee2df0c266e35f9f151e0e15a1, size: 27921326)
2021/05/22 12:11:06 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/22 12:11:06 control response channel closed
21/05/22 12:11:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa finished.
2021/05/22 12:11:07 Job state: DONE
21/05/22 12:11:07 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa
21/05/22 12:11:07 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621685461864309493-jenkins-0522121104-ac07f174_ac7a6249-1687-41d9-a676-12a47b8dbcaa
2021/05/22 12:11:07 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (26.35s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 26.369s
2021/05/22 12:11:08 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.018s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 43s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/f5ispndygi7xm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #4705
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/4705/display/redirect?page=changes>
Changes:
[Boyuan Zhang] [BEAM-10670] Update Twister2Runner.runTest with the same logic as
[noreply] Graph java 8 and 11 metrics separately (#14860)
------------------------------------------
[...truncated 564.16 KB...]
coders: <
key: "c3"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c2"
>
>
coders: <
key: "c4"
value: <
spec: <
urn: "beam:coder:global_window:v1"
>
>
>
coders: <
key: "c5"
value: <
spec: <
urn: "beam:coder:bytes:v1"
>
>
>
coders: <
key: "c6"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c2"
>
>
coders: <
key: "c7"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c0"
component_coder_ids: "c6"
>
>
coders: <
key: "c8"
value: <
spec: <
urn: "beam:coder:kv:v1"
>
component_coder_ids: "c2"
component_coder_ids: "c0"
>
>
coders: <
key: "c9"
value: <
spec: <
urn: "beam:coder:iterable:v1"
>
component_coder_ids: "c0"
>
>
environments: <
key: "go"
value: <
urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies: <
type_urn: "beam:artifact:type:go_worker_binary:v1"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "s2"
root_transform_ids: "e9"
root_transform_ids: "s6"
2021/05/22 06:12:08 Prepared job with id: go-job-5-1621663928151935543_5aab02b3-1d29-401d-9657-1a2d5c881837 and staging token: go-job-5-1621663928151935543_5aab02b3-1d29-401d-9657-1a2d5c881837
2021/05/22 06:12:08 Cross-compiling <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/integration/wordcount/wordcount_test.go> as /tmp/worker-5-1621663928156522893
21/05/22 06:12:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for go-job-5-1621663928151935543_5aab02b3-1d29-401d-9657-1a2d5c881837.
21/05/22 06:12:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for go-job-5-1621663928151935543_5aab02b3-1d29-401d-9657-1a2d5c881837.go.
21/05/22 06:12:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for go-job-5-1621663928151935543_5aab02b3-1d29-401d-9657-1a2d5c881837.null.
21/05/22 06:12:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for go-job-5-1621663928151935543_5aab02b3-1d29-401d-9657-1a2d5c881837.
2021/05/22 06:12:12 Staged binary artifact with token:
21/05/22 06:12:12 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa
21/05/22 06:12:12 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa
2021/05/22 06:12:12 Submitted job: go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa
2021/05/22 06:12:12 Job state: STOPPED
2021/05/22 06:12:12 Job state: STARTING
2021/05/22 06:12:12 Job state: RUNNING
21/05/22 06:12:12 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
21/05/22 06:12:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa on Spark master local
21/05/22 06:12:12 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
21/05/22 06:12:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa: Pipeline translated successfully. Computing outputs
21/05/22 06:12:13 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
21/05/22 06:12:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
21/05/22 06:12:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:44363 ...
21/05/22 06:12:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:32825 ...
21/05/22 06:12:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 38-1
21/05/22 06:12:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 38-2
21/05/22 06:12:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go>:504: Connecting via grpc @ localhost:35463 ...
21/05/22 06:12:14 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
21/05/22 06:12:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasource.go>:232: DataSource: 1 elements in 5008641 ns
21/05/22 06:12:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/temp_gopath/src/github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec/datasink.go>:84: DataSink: 0 elements in 2662467 ns
21/05/22 06:12:14 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v0"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\ng/tmp/beam-artifact-staging/31498646aed51313eaa3138c01724e0d453f87892701837b56b313e3926a89d5/1-go-worker"
role_urn: "beam:artifact:role:staging_to:v1"
role_payload: "\n\006worker"
}
21/05/22 06:12:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
21/05/22 06:12:14 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
21/05/22 06:12:14 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d0424af3eff6722d388d3d44e3de643facb3d5f8f33c9ee7908e0e407d2c4fbe. Logs:
2021/05/22 06:12:14 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-job-5-1621663928151935543"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"hooks" value:{string_value:"{}"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0job0501621663928151935543-jenkins-0522061212-9ffa586"}} fields:{key:"beam:option:options_id:v1" value:{number_value:13}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:32825"} artifact_endpoint:{url:"localhost:36989"} control_endpoint:{url:"localhost:44363"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\ng/tmp/beam-artifact-staging/31498646aed51313eaa3138c01724e0d453f87892701837b56b313e3926a89d5/1-go-worker" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\n\x06worker"}
2021/05/22 06:12:14 Initializing Go harness: /opt/apache/beam/boot --id=38-1 --provision_endpoint=localhost:44997
2021/05/22 06:12:14 Downloaded: /tmp/staged/worker (sha256: a60ca6db8d0402976b4bd40906afeb33d133e2391d644bd6fe5384471a5449e2, size: 27921326)
2021/05/22 06:12:14 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
Worker exited successfully!
2021/05/22 06:12:14 control response channel closed
21/05/22 06:12:15 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa finished.
2021/05/22 06:12:15 Job state: DONE
21/05/22 06:12:15 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa
21/05/22 06:12:15 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0job0501621663928151935543-jenkins-0522061212-9ffa586_c81deb84-e555-4f13-8f7a-e9be074b98aa
2021/05/22 06:12:15 Failed to deduce Step from MonitoringInfo: urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" payload:"\x01" labels:{key:"PCOLLECTION" value:"n1"}
--- PASS: TestWordCount (38.85s)
PASS
ok github.com/apache/beam/sdks/go/test/integration/wordcount 38.882s
2021/05/22 06:12:18 WARNING: proto: file "v1.proto" is already registered
previously from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/test/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
=== RUN TestXLang_Prefix
--- SKIP: TestXLang_Prefix (0.00s)
integration.go:134: Test TestXLang_Prefix is currently filtered for runner spark
=== RUN TestXLang_CoGroupBy
--- SKIP: TestXLang_CoGroupBy (0.00s)
integration.go:134: Test TestXLang_CoGroupBy is currently filtered for runner spark
=== RUN TestXLang_Combine
--- SKIP: TestXLang_Combine (0.00s)
integration.go:134: Test TestXLang_Combine is currently filtered for runner spark
=== RUN TestXLang_CombineGlobally
--- SKIP: TestXLang_CombineGlobally (0.00s)
integration.go:134: Test TestXLang_CombineGlobally is currently filtered for runner spark
=== RUN TestXLang_Flatten
--- SKIP: TestXLang_Flatten (0.00s)
integration.go:134: Test TestXLang_Flatten is currently filtered for runner spark
=== RUN TestXLang_GroupBy
--- SKIP: TestXLang_GroupBy (0.00s)
integration.go:134: Test TestXLang_GroupBy is currently filtered for runner spark
=== RUN TestXLang_Multi
--- SKIP: TestXLang_Multi (0.00s)
integration.go:134: Test TestXLang_Multi is currently filtered for runner spark
=== RUN TestXLang_Partition
--- SKIP: TestXLang_Partition (0.00s)
integration.go:134: Test TestXLang_Partition is currently filtered for runner spark
PASS
ok github.com/apache/beam/sdks/go/test/integration/xlang 0.038s
if [[ "$RUNNER" == "dataflow" ]]; then
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container"
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
if [[ -n "$EXPANSION_ADDR" ]]; then
# Delete the java cross-language container locally and remotely
docker rmi $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to remove container"
gcloud --quiet container images delete $JAVA_CONTAINER:$JAVA_TAG || echo "Failed to delete container"
fi
# Clean up tempdir
rm -rf $TMPDIR
fi
exit $TEST_EXIT_CODE
exit_background_processes
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 114
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 50s
79 actionable tasks: 52 executed, 27 from cache
Publishing build scan...
https://gradle.com/s/uiv33tsdsyqle
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org