You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/08/31 02:54:11 UTC
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6534
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6534/display/redirect?page=changes>
Changes:
[noreply] Minor: Fix option_from_runner_api typehint (#22946)
------------------------------------------
[...truncated 2.83 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 02:54:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 02:54:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 02:54:02 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 7d01bc762c3ff1fa74a3faad4c52c0627a7fc801fcc94952fc0d3acb8f8da4d0. Logs:
2022/08/31 02:54:02 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-164"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46327"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test11705"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test11705"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33419"} artifact_endpoint:{url:"localhost:36067"} control_endpoint:{url:"localhost:42255"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 02:54:02 Initializing Go harness: /opt/apache/beam/boot --id=275-1 --provision_endpoint=localhost:35253
2022/08/31 02:54:02 Downloaded: /tmp/staged/1-worker-13-1661914438253161746 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 02:54:02 control response channel closed
22/08/31 02:54:03 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 02:54:03 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 02:54:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 02:54:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 276-1
22/08/31 02:54:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34463 for control ...
22/08/31 02:54:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33659 for logging ...
22/08/31 02:54:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 276-2
22/08/31 02:54:04 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39699 for data ...
22/08/31 02:54:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 02:54:04 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 02:54:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 02:54:04 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 02:54:04 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d69f16bac324640a14166480d966344482474abca030dd3256a0dbdb46b819c9. Logs:
2022/08/31 02:54:04 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-164"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46327"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test11705"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test11705"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33659"} artifact_endpoint:{url:"localhost:36805"} control_endpoint:{url:"localhost:34463"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 02:54:04 Initializing Go harness: /opt/apache/beam/boot --id=276-1 --provision_endpoint=localhost:41583
2022/08/31 02:54:04 Downloaded: /tmp/staged/1-worker-13-1661914438253161746 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
2022/08/31 02:54:04 control response channel closed
Worker exited successfully!
22/08/31 02:54:04 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 02:54:05 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 02:54:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 02:54:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 277-1
22/08/31 02:54:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44981 for control ...
22/08/31 02:54:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45115 for logging ...
22/08/31 02:54:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 277-2
22/08/31 02:54:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46005 for data ...
22/08/31 02:54:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 02:54:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44491 for state ...
22/08/31 02:54:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 02:54:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 02:54:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 02:54:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container bf6ad9198abdafb8a22840d4d584fe4e52401a208086a668e1b1ad9accde28ad. Logs:
2022/08/31 02:54:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-164"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46327"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test11705"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test11705"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45115"} artifact_endpoint:{url:"localhost:45593"} control_endpoint:{url:"localhost:44981"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 02:54:05 Initializing Go harness: /opt/apache/beam/boot --id=277-1 --provision_endpoint=localhost:37339
2022/08/31 02:54:05 Downloaded: /tmp/staged/1-worker-13-1661914438253161746 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
2022/08/31 02:54:05 control response channel closed
Worker exited successfully!
22/08/31 02:54:06 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 02:54:06 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 02:54:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7_2de44270-02ea-40b7-89e7-bea93d795eed: Pipeline translated successfully. Computing outputs
22/08/31 02:54:06 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 02:54:07 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 02:54:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 278-1
22/08/31 02:54:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 02:54:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:32795 for control ...
22/08/31 02:54:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34257 for logging ...
22/08/31 02:54:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 278-2
22/08/31 02:54:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40621 for data ...
22/08/31 02:54:07 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 02:54:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41473 for state ...
22/08/31 02:54:07 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 02:54:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 02:54:07 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 02:54:07 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 64d403642e8e39c07813daecc3ceb0be538ad03374d2842b9693a1c1c3364d00. Logs:
2022/08/31 02:54:07 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-164"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46327"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test11705"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test11705"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34257"} artifact_endpoint:{url:"localhost:46039"} control_endpoint:{url:"localhost:32795"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/1-go-/tmp/worker-13-1661914438253161746" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 02:54:07 Initializing Go harness: /opt/apache/beam/boot --id=278-1 --provision_endpoint=localhost:35975
2022/08/31 02:54:07 Downloaded: /tmp/staged/1-worker-13-1661914438253161746 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 02:54:07 control response channel closed
22/08/31 02:54:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7_2de44270-02ea-40b7-89e7-bea93d795eed finished.
22/08/31 02:54:08 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-164_b8c2de05-5406-49ee-ace9-da4eecc0d3c5.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/3bcc8cea93aecbdd414d4ad0a5d9efa4fef0d3fb2177c0664623aa98465dee66/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/08/31 02:54:08 Job state: DONE
22/08/31 02:54:08 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7_2de44270-02ea-40b7-89e7-bea93d795eed
22/08/31 02:54:08 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0164-jenkins-0831025359-f6b8a4a7_2de44270-02ea-40b7-89e7-bea93d795eed
--- PASS: TestEmitParDoAfterGBK (9.85s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 112.508s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 14m 42s
114 actionable tasks: 80 executed, 29 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/22grrdvpqnrvk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Go_VR_Spark #6562
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6562/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6561/display/redirect?page=changes>
Changes:
[noreply] Allow users to pass classloader to dynamically load JDBC drivers.
[noreply] Fix withCheckStopReadingFn to not cause the pipeline to crash (#22962)
[noreply] Inference benchmark tests (#21738)
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 14:53:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 14:53:42 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 14:53:42 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 7603aac2ae2e44a96f3fdfa53bf0ad45af4b0f7595c636125a9ac5ade734fb4d. Logs:
2022/09/06 14:53:42 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-287"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:49287"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test25000"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test25000"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41911"} artifact_endpoint:{url:"localhost:33337"} control_endpoint:{url:"localhost:34083"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 14:53:42 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:36327
2022/09/06 14:53:42 Downloaded: /tmp/staged/1-worker-13-1662476017647856611 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 14:53:42 control response channel closed
22/09/06 14:53:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 14:53:43 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 14:53:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 14:53:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/06 14:53:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35033 for control ...
22/09/06 14:53:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46169 for logging ...
22/09/06 14:53:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/06 14:53:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38335 for data ...
22/09/06 14:53:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 14:53:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 14:53:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 14:53:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 14:53:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 90b90f1252df1e871f8176664dcf074414af74badf91c9d33f1f5a02e8459958. Logs:
2022/09/06 14:53:44 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-287"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:49287"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test25000"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test25000"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46169"} artifact_endpoint:{url:"localhost:45059"} control_endpoint:{url:"localhost:35033"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 14:53:44 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:35657
2022/09/06 14:53:44 Downloaded: /tmp/staged/1-worker-13-1662476017647856611 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 14:53:44 control response channel closed
22/09/06 14:53:45 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 14:53:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 14:53:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 14:53:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/06 14:53:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35787 for control ...
22/09/06 14:53:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38407 for logging ...
22/09/06 14:53:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/06 14:53:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44471 for data ...
22/09/06 14:53:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 14:53:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39127 for state ...
22/09/06 14:53:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 14:53:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 14:53:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 14:53:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ddf43f9c6e774b16d17b732ac684888da8a347a2e5b66951094a3d09f8be4823. Logs:
2022/09/06 14:53:45 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-287"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:49287"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test25000"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test25000"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38407"} artifact_endpoint:{url:"localhost:36023"} control_endpoint:{url:"localhost:35787"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 14:53:45 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:38867
2022/09/06 14:53:45 Downloaded: /tmp/staged/1-worker-13-1662476017647856611 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/06 14:53:46 control response channel closed
Worker exited successfully!
22/09/06 14:53:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 14:53:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 14:53:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e_e67ae1dc-d465-471b-a90a-f9e1918411b3: Pipeline translated successfully. Computing outputs
22/09/06 14:53:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 14:53:47 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 14:53:47 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 14:53:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/06 14:53:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36053 for control ...
22/09/06 14:53:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42985 for logging ...
22/09/06 14:53:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/06 14:53:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37511 for data ...
22/09/06 14:53:47 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 14:53:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39329 for state ...
22/09/06 14:53:47 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 14:53:47 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 14:53:47 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 14:53:47 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 3a020084ecc871f81c10bb557cfe388d9e4924a6e9ef060d6472abfe9fd43588. Logs:
2022/09/06 14:53:47 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-287"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:49287"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test25000"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test25000"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42985"} artifact_endpoint:{url:"localhost:33339"} control_endpoint:{url:"localhost:36053"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/1-go-/tmp/worker-13-1662476017647856611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 14:53:47 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:42791
2022/09/06 14:53:47 Downloaded: /tmp/staged/1-worker-13-1662476017647856611 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 14:53:47 control response channel closed
22/09/06 14:53:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e_e67ae1dc-d465-471b-a90a-f9e1918411b3 finished.
22/09/06 14:53:48 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-287_85fad8e2-9030-45d4-aa24-1c8a61b78664.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/9fb28019465e0ce5339060c5617a4319f696cfc18e13db766cdd9f2b35ef7516/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/06 14:53:48 Job state: DONE
22/09/06 14:53:48 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e_e67ae1dc-d465-471b-a90a-f9e1918411b3
22/09/06 14:53:48 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0287-jenkins-0906145339-e668d96e_e67ae1dc-d465-471b-a90a-f9e1918411b3
--- PASS: TestEmitParDoAfterGBK (10.75s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 118.539s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 14m 23s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/r2ocx4ghqrb7u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6560/display/redirect>
Changes:
------------------------------------------
[...truncated 2.51 MB...]
22/09/06 08:52:31 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36363 for data ...
22/09/06 08:52:31 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 08:52:31 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 08:52:31 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 08:52:31 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 08:52:31 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ce2798431da87303a1acdbc82f94db7b8d4c931fd487ddbe44895ed35dddc118. Logs:
2022/09/06 08:52:31 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-364"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:45367"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test9429"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test9429"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41077"} artifact_endpoint:{url:"localhost:38057"} control_endpoint:{url:"localhost:36173"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 08:52:31 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:39747
2022/09/06 08:52:31 Downloaded: /tmp/staged/1-worker-13-1662454347216129007 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/06 08:52:31 control response channel closed
Worker exited successfully!
22/09/06 08:52:32 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 08:52:33 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 08:52:33 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 08:52:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/06 08:52:33 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40533 for control ...
22/09/06 08:52:33 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38605 for logging ...
22/09/06 08:52:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/06 08:52:33 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42043 for data ...
22/09/06 08:52:33 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 08:52:33 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 08:52:33 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 08:52:33 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 08:52:33 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 8967dccea9d601ea86a9b7617e4ed07d5f1e08a0ee78eb0d661dcd1f151ef1ca. Logs:
2022/09/06 08:52:33 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-364"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:45367"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test9429"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test9429"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38605"} artifact_endpoint:{url:"localhost:37755"} control_endpoint:{url:"localhost:40533"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 08:52:33 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:43907
2022/09/06 08:52:33 Downloaded: /tmp/staged/1-worker-13-1662454347216129007 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 08:52:33 control response channel closed
22/09/06 08:52:34 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 08:52:34 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 08:52:35 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 08:52:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/06 08:52:35 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42239 for control ...
22/09/06 08:52:35 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39763 for logging ...
22/09/06 08:52:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/06 08:52:35 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34185 for data ...
22/09/06 08:52:35 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 08:52:35 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36513 for state ...
22/09/06 08:52:35 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 08:52:35 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 08:52:35 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 08:52:35 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9b0b7550ddbcf5f25dcba2b75fc116bcb7cd1a45f1d405a8353c5083b56cfd8f. Logs:
2022/09/06 08:52:34 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-364"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:45367"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test9429"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test9429"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39763"} artifact_endpoint:{url:"localhost:41955"} control_endpoint:{url:"localhost:42239"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 08:52:34 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:35229
2022/09/06 08:52:35 Downloaded: /tmp/staged/1-worker-13-1662454347216129007 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 08:52:35 control response channel closed
22/09/06 08:52:35 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 08:52:35 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 08:52:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f_058bcd1a-a18f-4d3d-811b-0a2713f87418: Pipeline translated successfully. Computing outputs
22/09/06 08:52:35 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 08:52:36 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 08:52:36 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 08:52:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/06 08:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34173 for control ...
22/09/06 08:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34777 for logging ...
22/09/06 08:52:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/06 08:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39685 for data ...
22/09/06 08:52:36 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 08:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34659 for state ...
22/09/06 08:52:36 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 08:52:36 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 08:52:36 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 08:52:36 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 5bf2d0596fc68de76d299d2ea9960c43cf8b25e764c1ef224170e8fb9fc2178a. Logs:
2022/09/06 08:52:36 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-364"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:45367"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test9429"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test9429"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34777"} artifact_endpoint:{url:"localhost:34713"} control_endpoint:{url:"localhost:34173"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/1-go-/tmp/worker-13-1662454347216129007" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 08:52:36 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:37513
2022/09/06 08:52:36 Downloaded: /tmp/staged/1-worker-13-1662454347216129007 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 08:52:36 control response channel closed
22/09/06 08:52:37 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f_058bcd1a-a18f-4d3d-811b-0a2713f87418 finished.
22/09/06 08:52:37 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-364_13d08690-4c83-44ba-9f44-728df1bc8ad7.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/9efbf595c89e41b21d8fdb7ea4876e5adfd51056eaa9f7ac473f247e5e2ec35a/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/06 08:52:37 Job state: DONE
22/09/06 08:52:37 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f_058bcd1a-a18f-4d3d-811b-0a2713f87418
22/09/06 08:52:37 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0364-jenkins-0906085228-85c8d52f_058bcd1a-a18f-4d3d-811b-0a2713f87418
--- PASS: TestEmitParDoAfterGBK (10.13s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 113.200s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 11s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/sblgbckp7mmui
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6559/display/redirect?page=changes>
Changes:
[noreply] Revert "Remove subprocess.PIPE usage by using a temp file (#22654)"
------------------------------------------
[...truncated 2.51 MB...]
22/09/06 02:52:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37303 for data ...
22/09/06 02:52:18 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 02:52:18 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 02:52:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 02:52:18 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 02:52:18 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 089f0955bc7c1198ccfa3489d0efe8e071681c8213beaa8fe2a2049b2bd0ccdf. Logs:
2022/09/06 02:52:18 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-587"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42407"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test17976"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test17976"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38325"} artifact_endpoint:{url:"localhost:34659"} control_endpoint:{url:"localhost:45385"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 02:52:18 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:41937
2022/09/06 02:52:18 Downloaded: /tmp/staged/1-worker-13-1662432733696659296 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 02:52:18 control response channel closed
22/09/06 02:52:19 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 02:52:19 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 02:52:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/06 02:52:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 02:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39193 for control ...
22/09/06 02:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46737 for logging ...
22/09/06 02:52:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/06 02:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46471 for data ...
22/09/06 02:52:19 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 02:52:20 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 02:52:20 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 02:52:20 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 02:52:20 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9b1a523a58cee99876160531ca3b2f0bd06c7b110f3e691a71079bb982834f6c. Logs:
2022/09/06 02:52:19 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-587"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42407"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test17976"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test17976"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46737"} artifact_endpoint:{url:"localhost:42395"} control_endpoint:{url:"localhost:39193"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 02:52:19 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:37841
2022/09/06 02:52:19 Downloaded: /tmp/staged/1-worker-13-1662432733696659296 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/06 02:52:20 control response channel closed
Worker exited successfully!
22/09/06 02:52:20 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 02:52:21 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 02:52:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 02:52:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/06 02:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36591 for control ...
22/09/06 02:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45735 for logging ...
22/09/06 02:52:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/06 02:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41647 for data ...
22/09/06 02:52:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 02:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33841 for state ...
22/09/06 02:52:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 02:52:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 02:52:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 02:52:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 72ec38a87b8825132e34ac70d6de656629b5b41efa7d51d3f6a2ee7200cc3fb4. Logs:
2022/09/06 02:52:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-587"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42407"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test17976"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test17976"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45735"} artifact_endpoint:{url:"localhost:42721"} control_endpoint:{url:"localhost:36591"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 02:52:21 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:36725
2022/09/06 02:52:21 Downloaded: /tmp/staged/1-worker-13-1662432733696659296 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 02:52:21 control response channel closed
22/09/06 02:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 02:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 02:52:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88_09ea6193-9180-4679-a49c-c0c4d6e227d8: Pipeline translated successfully. Computing outputs
22/09/06 02:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/06 02:52:22 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/06 02:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/06 02:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/06 02:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41063 for control ...
22/09/06 02:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33711 for logging ...
22/09/06 02:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/06 02:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37813 for data ...
22/09/06 02:52:23 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/06 02:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33155 for state ...
22/09/06 02:52:23 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/06 02:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/06 02:52:23 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/06 02:52:23 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 08af0a262c51c323c335b8ff6d801d0853c10a996f1156fceb47b873a1cb18d7. Logs:
2022/09/06 02:52:22 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-587"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42407"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test17976"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test17976"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33711"} artifact_endpoint:{url:"localhost:36637"} control_endpoint:{url:"localhost:41063"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/1-go-/tmp/worker-13-1662432733696659296" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/06 02:52:22 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:35295
2022/09/06 02:52:23 Downloaded: /tmp/staged/1-worker-13-1662432733696659296 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/06 02:52:23 control response channel closed
22/09/06 02:52:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88_09ea6193-9180-4679-a49c-c0c4d6e227d8 finished.
22/09/06 02:52:23 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-587_14e83645-4aa4-49ca-8ba4-ba936bd2ab70.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/a58f0f22a3eb62594eaa75a26e8cbbe6abe4c98cb976fd628b495b49f356af38/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/06 02:52:23 Job state: DONE
22/09/06 02:52:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88_09ea6193-9180-4679-a49c-c0c4d6e227d8
22/09/06 02:52:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0587-jenkins-0906025215-9e7b9c88_09ea6193-9180-4679-a49c-c0c4d6e227d8
--- PASS: TestEmitParDoAfterGBK (10.15s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 111.684s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 12m 58s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/ewx266dyubhhe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6558/display/redirect>
Changes:
------------------------------------------
[...truncated 2.52 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 20:52:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 20:52:18 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 20:52:18 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 4796b0225fd81c62f9e9568dfcbf3dcba5cf9cb025e3179539532dc522fe7e40. Logs:
2022/09/05 20:52:18 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-837"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1422"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1422"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38181"} artifact_endpoint:{url:"localhost:42765"} control_endpoint:{url:"localhost:33873"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 20:52:18 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:36169
2022/09/05 20:52:18 Downloaded: /tmp/staged/1-worker-13-1662411133563470709 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 20:52:18 control response channel closed
22/09/05 20:52:18 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 20:52:19 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 20:52:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 20:52:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/05 20:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39499 for control ...
22/09/05 20:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45921 for logging ...
22/09/05 20:52:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/05 20:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34613 for data ...
22/09/05 20:52:19 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 20:52:19 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 20:52:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 20:52:19 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 20:52:20 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ae60f250a13fdddc14915fdda7008cfd8d7e21976cf10cd894d4ee53fa6c414f. Logs:
2022/09/05 20:52:19 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-837"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1422"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1422"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45921"} artifact_endpoint:{url:"localhost:46199"} control_endpoint:{url:"localhost:39499"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 20:52:19 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:37167
2022/09/05 20:52:19 Downloaded: /tmp/staged/1-worker-13-1662411133563470709 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/05 20:52:19 control response channel closed
Worker exited successfully!
22/09/05 20:52:20 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 20:52:21 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 20:52:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 20:52:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/05 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41941 for control ...
22/09/05 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33587 for logging ...
22/09/05 20:52:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/05 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37769 for data ...
22/09/05 20:52:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40135 for state ...
22/09/05 20:52:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 20:52:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 20:52:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 20:52:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b3fb59fb56f02683890a705168eb67114ffc85e6db13836036b9e4c0e677aa43. Logs:
2022/09/05 20:52:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-837"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1422"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1422"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33587"} artifact_endpoint:{url:"localhost:37473"} control_endpoint:{url:"localhost:41941"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 20:52:21 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:38751
2022/09/05 20:52:21 Downloaded: /tmp/staged/1-worker-13-1662411133563470709 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 20:52:21 control response channel closed
22/09/05 20:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 20:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 20:52:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901_3eb4bbea-aa63-49e1-b0fe-2dfb63971f7d: Pipeline translated successfully. Computing outputs
22/09/05 20:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 20:52:22 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 20:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/05 20:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44695 for control ...
22/09/05 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36305 for logging ...
22/09/05 20:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/05 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33859 for data ...
22/09/05 20:52:23 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36651 for state ...
22/09/05 20:52:23 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 20:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 20:52:23 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 20:52:23 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6f6d8ab9f80bab9a4ecb7115671304b335f6f01b48787401b0746e82031d7bfa. Logs:
2022/09/05 20:52:22 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-837"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:42093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1422"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1422"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36305"} artifact_endpoint:{url:"localhost:35857"} control_endpoint:{url:"localhost:44695"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/1-go-/tmp/worker-13-1662411133563470709" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 20:52:22 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:40335
2022/09/05 20:52:23 Downloaded: /tmp/staged/1-worker-13-1662411133563470709 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 20:52:23 control response channel closed
22/09/05 20:52:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901_3eb4bbea-aa63-49e1-b0fe-2dfb63971f7d finished.
22/09/05 20:52:23 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-837_a47956d0-41e9-4915-a389-e0db84478399.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/f951987054cb3b26fca95e75aac3c51b6c55fd89288b22c37d1fd28647a14afe/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/05 20:52:23 Job state: DONE
22/09/05 20:52:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901_3eb4bbea-aa63-49e1-b0fe-2dfb63971f7d
22/09/05 20:52:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0837-jenkins-0905205215-78a40901_3eb4bbea-aa63-49e1-b0fe-2dfb63971f7d
--- PASS: TestEmitParDoAfterGBK (10.22s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 111.500s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 12m 58s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/uzhlgdpna3tnw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6557
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6557/display/redirect>
Changes:
------------------------------------------
[...truncated 2.52 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 14:52:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 14:52:42 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 14:52:42 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 41bd6c5f38924a483f6b1035d3515f090e36ea15421acf0ce335b09dafd5df85. Logs:
2022/09/05 14:52:42 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-377"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38481"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test30869"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test30869"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46171"} artifact_endpoint:{url:"localhost:39477"} control_endpoint:{url:"localhost:41711"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 14:52:42 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:39137
2022/09/05 14:52:42 Downloaded: /tmp/staged/1-worker-13-1662389557346430923 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 14:52:42 control response channel closed
22/09/05 14:52:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 14:52:43 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 14:52:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 14:52:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/05 14:52:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40905 for control ...
22/09/05 14:52:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42069 for logging ...
22/09/05 14:52:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/05 14:52:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34461 for data ...
22/09/05 14:52:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 14:52:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 14:52:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 14:52:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 14:52:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 4cd53e920f4542e5992204fc867b056a71b33dedb97e3431394104253edbc7d5. Logs:
2022/09/05 14:52:43 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-377"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38481"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test30869"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test30869"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42069"} artifact_endpoint:{url:"localhost:38295"} control_endpoint:{url:"localhost:40905"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 14:52:43 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:45151
2022/09/05 14:52:43 Downloaded: /tmp/staged/1-worker-13-1662389557346430923 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 14:52:44 control response channel closed
22/09/05 14:52:44 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 14:52:45 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 14:52:45 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 14:52:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/05 14:52:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45957 for control ...
22/09/05 14:52:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46605 for logging ...
22/09/05 14:52:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/05 14:52:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33083 for data ...
22/09/05 14:52:45 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 14:52:45 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46251 for state ...
22/09/05 14:52:45 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 14:52:45 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 14:52:45 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 14:52:45 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 7a6ef20d8cfc050003e9d51ed9a78ec82af15bf1d7d0aba5fb3398a9478ebcee. Logs:
2022/09/05 14:52:45 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-377"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38481"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test30869"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test30869"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46605"} artifact_endpoint:{url:"localhost:40207"} control_endpoint:{url:"localhost:45957"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 14:52:45 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:33257
2022/09/05 14:52:45 Downloaded: /tmp/staged/1-worker-13-1662389557346430923 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 14:52:45 control response channel closed
22/09/05 14:52:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 14:52:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 14:52:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6_4770f1ff-ac66-454b-b516-43b3b9819b80: Pipeline translated successfully. Computing outputs
22/09/05 14:52:46 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 14:52:46 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 14:52:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/05 14:52:47 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 14:52:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39191 for control ...
22/09/05 14:52:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42695 for logging ...
22/09/05 14:52:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/05 14:52:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34393 for data ...
22/09/05 14:52:47 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 14:52:47 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42093 for state ...
22/09/05 14:52:47 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 14:52:47 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 14:52:47 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 14:52:47 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 3719d41d43e2a5fc5c88da99da3914e864a6ecf3d98fb246b7950496e1295cd3. Logs:
2022/09/05 14:52:47 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-377"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38481"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test30869"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test30869"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42695"} artifact_endpoint:{url:"localhost:33651"} control_endpoint:{url:"localhost:39191"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/1-go-/tmp/worker-13-1662389557346430923" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 14:52:47 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:40485
2022/09/05 14:52:47 Downloaded: /tmp/staged/1-worker-13-1662389557346430923 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 14:52:47 control response channel closed
22/09/05 14:52:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6_4770f1ff-ac66-454b-b516-43b3b9819b80 finished.
22/09/05 14:52:47 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-377_10178982-e8c4-4af5-a6d3-815550bd60fc.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/dd1a0a258fc5b3c52b7e78a89ece6b56dfd5efb140a748c936a91fc2812a0430/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/05 14:52:47 Job state: DONE
22/09/05 14:52:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6_4770f1ff-ac66-454b-b516-43b3b9819b80
22/09/05 14:52:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0377-jenkins-0905145239-4c35b4e6_4770f1ff-ac66-454b-b516-43b3b9819b80
--- PASS: TestEmitParDoAfterGBK (10.54s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 112.688s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 23s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/x42wky4w7hfey
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6556
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6556/display/redirect?page=changes>
Changes:
[noreply] Generalize interface of InfluxDBPublisher to support more use cases
------------------------------------------
[...truncated 2.52 MB...]
22/09/05 08:53:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37761 for data ...
22/09/05 08:53:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 08:53:12 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 08:53:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 08:53:12 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 08:53:12 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 08404c9277fe8daaa3b1809bf1425212c3443fb62d9fa255ab4837b13648488d. Logs:
2022/09/05 08:53:12 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-657"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33487"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29434"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29434"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33173"} artifact_endpoint:{url:"localhost:36641"} control_endpoint:{url:"localhost:45017"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 08:53:12 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:36077
2022/09/05 08:53:12 Downloaded: /tmp/staged/1-worker-13-1662367987849325587 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/05 08:53:12 control response channel closed
Worker exited successfully!
22/09/05 08:53:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 08:53:13 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 08:53:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 08:53:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/05 08:53:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38275 for control ...
22/09/05 08:53:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44023 for logging ...
22/09/05 08:53:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/05 08:53:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41251 for data ...
22/09/05 08:53:14 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 08:53:14 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 08:53:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 08:53:14 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 08:53:14 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 51a4f2c406930847c1fed93e55542d3ab89cfc60fc50649067688692ef9881bd. Logs:
2022/09/05 08:53:14 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-657"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33487"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29434"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29434"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44023"} artifact_endpoint:{url:"localhost:44407"} control_endpoint:{url:"localhost:38275"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 08:53:14 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:39605
2022/09/05 08:53:14 Downloaded: /tmp/staged/1-worker-13-1662367987849325587 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/05 08:53:14 control response channel closed
Worker exited successfully!
22/09/05 08:53:14 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 08:53:15 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 08:53:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/05 08:53:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 08:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34529 for control ...
22/09/05 08:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39477 for logging ...
22/09/05 08:53:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/05 08:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42367 for data ...
22/09/05 08:53:15 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 08:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45331 for state ...
22/09/05 08:53:15 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 08:53:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 08:53:15 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 08:53:15 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f8241cbaf1fd75e806441737846d58bd0e11200e72e640fc8a98b9760700de3e. Logs:
2022/09/05 08:53:15 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-657"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33487"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29434"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29434"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39477"} artifact_endpoint:{url:"localhost:44719"} control_endpoint:{url:"localhost:34529"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 08:53:15 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:33089
2022/09/05 08:53:15 Downloaded: /tmp/staged/1-worker-13-1662367987849325587 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 08:53:15 control response channel closed
22/09/05 08:53:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 08:53:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 08:53:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a_bf4585c8-df98-47bc-8358-c00c7b720ec1: Pipeline translated successfully. Computing outputs
22/09/05 08:53:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 08:53:17 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 08:53:17 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 08:53:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/05 08:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36967 for control ...
22/09/05 08:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39679 for logging ...
22/09/05 08:53:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/05 08:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33469 for data ...
22/09/05 08:53:17 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 08:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46505 for state ...
22/09/05 08:53:17 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 08:53:17 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 08:53:17 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 08:53:17 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 2af53c3a37a5eacdb3dc0cd4ddc4163307f922ec538ad4b8eb95a5ea0d58199f. Logs:
2022/09/05 08:53:17 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-657"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33487"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29434"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29434"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39679"} artifact_endpoint:{url:"localhost:33683"} control_endpoint:{url:"localhost:36967"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/1-go-/tmp/worker-13-1662367987849325587" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 08:53:17 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:36423
2022/09/05 08:53:17 Downloaded: /tmp/staged/1-worker-13-1662367987849325587 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/05 08:53:17 control response channel closed
Worker exited successfully!
22/09/05 08:53:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a_bf4585c8-df98-47bc-8358-c00c7b720ec1 finished.
22/09/05 08:53:18 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-657_527417b9-da00-40ea-bc2a-070767090575.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/2dbf8343fe05aeefd844c8f3d32aa95aec87db8543047bd80b786af50d9b56fc/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/05 08:53:18 Job state: DONE
22/09/05 08:53:18 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a_bf4585c8-df98-47bc-8358-c00c7b720ec1
22/09/05 08:53:18 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0657-jenkins-0905085309-8e161c5a_bf4585c8-df98-47bc-8358-c00c7b720ec1
--- PASS: TestEmitParDoAfterGBK (10.31s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 115.324s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 42s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/hgfyswn5aimtg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6555
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6555/display/redirect>
Changes:
------------------------------------------
[...truncated 2.51 MB...]
22/09/05 02:51:58 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33423 for data ...
22/09/05 02:51:58 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 02:51:58 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 02:51:58 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 02:51:58 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 02:51:58 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f32441dbd8088114ddf1ee8502ead7c2dabb092b964263fe4384c5f0f81fd7ac. Logs:
2022/09/05 02:51:57 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-975"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50961"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test28357"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test28357"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40425"} artifact_endpoint:{url:"localhost:40619"} control_endpoint:{url:"localhost:37561"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 02:51:57 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:36879
2022/09/05 02:51:58 Downloaded: /tmp/staged/1-worker-13-1662346313432677769 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/05 02:51:58 control response channel closed
Worker exited successfully!
22/09/05 02:51:58 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 02:51:59 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 02:51:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/05 02:51:59 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 02:51:59 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38767 for control ...
22/09/05 02:51:59 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34237 for logging ...
22/09/05 02:51:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/05 02:51:59 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45573 for data ...
22/09/05 02:51:59 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 02:51:59 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 02:51:59 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 02:51:59 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 02:51:59 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 30a75b4287b06bac32b0c7a96c5d1ad29f89411ef6ac1a101785bddba058f9b5. Logs:
2022/09/05 02:51:59 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-975"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50961"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test28357"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test28357"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34237"} artifact_endpoint:{url:"localhost:45691"} control_endpoint:{url:"localhost:38767"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 02:51:59 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:45891
2022/09/05 02:51:59 Downloaded: /tmp/staged/1-worker-13-1662346313432677769 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 02:51:59 control response channel closed
22/09/05 02:52:00 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 02:52:00 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 02:52:01 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 02:52:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/05 02:52:01 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44357 for control ...
22/09/05 02:52:01 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34165 for logging ...
22/09/05 02:52:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/05 02:52:01 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46817 for data ...
22/09/05 02:52:01 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 02:52:01 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41857 for state ...
22/09/05 02:52:01 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 02:52:01 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 02:52:01 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 02:52:01 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 605aaf5a84549875c3db4b7b98eb548f8bd18d33836bb8886004670060d508b9. Logs:
2022/09/05 02:52:01 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-975"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50961"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test28357"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test28357"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34165"} artifact_endpoint:{url:"localhost:41521"} control_endpoint:{url:"localhost:44357"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 02:52:01 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:43927
2022/09/05 02:52:01 Downloaded: /tmp/staged/1-worker-13-1662346313432677769 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 02:52:01 control response channel closed
22/09/05 02:52:01 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 02:52:02 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 02:52:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948_ffe78b89-88aa-44ea-834d-1028f9cde3e0: Pipeline translated successfully. Computing outputs
22/09/05 02:52:02 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/05 02:52:02 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/05 02:52:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/05 02:52:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/05 02:52:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39275 for control ...
22/09/05 02:52:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42325 for logging ...
22/09/05 02:52:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/05 02:52:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35611 for data ...
22/09/05 02:52:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/05 02:52:02 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40507 for state ...
22/09/05 02:52:02 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/05 02:52:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/05 02:52:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/05 02:52:03 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 5eea0b000fc73f6f76bf0fa2f69e53cff0704c505bd7d4d4795bab5ace50a422. Logs:
2022/09/05 02:52:02 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-975"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50961"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test28357"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test28357"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42325"} artifact_endpoint:{url:"localhost:37067"} control_endpoint:{url:"localhost:39275"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/1-go-/tmp/worker-13-1662346313432677769" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/05 02:52:02 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:40623
2022/09/05 02:52:02 Downloaded: /tmp/staged/1-worker-13-1662346313432677769 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/05 02:52:02 control response channel closed
22/09/05 02:52:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948_ffe78b89-88aa-44ea-834d-1028f9cde3e0 finished.
22/09/05 02:52:03 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-975_8326d65e-be3a-4e95-a8d6-d83537efe13a.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/77fe390c3036e75bdf2eda7e6c575a7cf6304b79dffb2142f68d18812edf04b4/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/05 02:52:03 Job state: DONE
22/09/05 02:52:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948_ffe78b89-88aa-44ea-834d-1028f9cde3e0
22/09/05 02:52:03 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0975-jenkins-0905025155-a8606948_ffe78b89-88aa-44ea-834d-1028f9cde3e0
--- PASS: TestEmitParDoAfterGBK (10.15s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 110.821s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 12m 36s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/xgt47txy6nww2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6554
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6554/display/redirect>
Changes:
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 20:52:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 20:52:41 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 20:52:41 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 7b48378f54254dc2683cb9655fd392cba20f21a454fcd57653e5afc94b93853c. Logs:
2022/09/04 20:52:41 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-600"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50435"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7744"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7744"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33207"} artifact_endpoint:{url:"localhost:43907"} control_endpoint:{url:"localhost:42649"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 20:52:41 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:44161
2022/09/04 20:52:41 Downloaded: /tmp/staged/1-worker-13-1662324757048855472 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/04 20:52:41 control response channel closed
Worker exited successfully!
22/09/04 20:52:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 20:52:42 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 20:52:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/04 20:52:43 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 20:52:43 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34553 for control ...
22/09/04 20:52:43 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43869 for logging ...
22/09/04 20:52:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/04 20:52:43 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46245 for data ...
22/09/04 20:52:43 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 20:52:43 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 20:52:43 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 20:52:43 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 20:52:43 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 424de521f042726e5f611cf4206c28b522c998454bcbc9ab8e8ea3305e7656e5. Logs:
2022/09/04 20:52:43 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-600"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50435"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7744"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7744"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43869"} artifact_endpoint:{url:"localhost:36693"} control_endpoint:{url:"localhost:34553"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 20:52:43 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:36721
2022/09/04 20:52:43 Downloaded: /tmp/staged/1-worker-13-1662324757048855472 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 20:52:43 control response channel closed
22/09/04 20:52:43 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 20:52:44 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 20:52:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 20:52:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/04 20:52:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38721 for control ...
22/09/04 20:52:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46367 for logging ...
22/09/04 20:52:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/04 20:52:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36141 for data ...
22/09/04 20:52:44 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 20:52:44 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38703 for state ...
22/09/04 20:52:44 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 20:52:44 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 20:52:44 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 20:52:44 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6636fb6a57ad3863a9e78e0e19df54ff28b279caf6209cbe1a13c51fbc4ea91b. Logs:
2022/09/04 20:52:44 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-600"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50435"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7744"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7744"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46367"} artifact_endpoint:{url:"localhost:46523"} control_endpoint:{url:"localhost:38721"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 20:52:44 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:45057
2022/09/04 20:52:44 Downloaded: /tmp/staged/1-worker-13-1662324757048855472 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 20:52:44 control response channel closed
22/09/04 20:52:45 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 20:52:45 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 20:52:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd_44e33d5f-c471-462e-8bd1-3e8616cedf6b: Pipeline translated successfully. Computing outputs
22/09/04 20:52:45 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 20:52:46 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 20:52:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 20:52:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/04 20:52:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43003 for control ...
22/09/04 20:52:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35569 for logging ...
22/09/04 20:52:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/04 20:52:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34687 for data ...
22/09/04 20:52:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 20:52:46 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35723 for state ...
22/09/04 20:52:46 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 20:52:46 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 20:52:46 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 20:52:46 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 16c1e155bbeecf523b9b78937709c7edb6b381cb046a89a45951f5f465c4a3ab. Logs:
2022/09/04 20:52:46 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-600"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:50435"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7744"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7744"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35569"} artifact_endpoint:{url:"localhost:37081"} control_endpoint:{url:"localhost:43003"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/1-go-/tmp/worker-13-1662324757048855472" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 20:52:46 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:34833
2022/09/04 20:52:46 Downloaded: /tmp/staged/1-worker-13-1662324757048855472 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 20:52:46 control response channel closed
22/09/04 20:52:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd_44e33d5f-c471-462e-8bd1-3e8616cedf6b finished.
22/09/04 20:52:47 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-600_378d1ae9-3cdc-49be-9e0c-453fc8840ebe.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/b14a775a433383dbac2b2f1fe1914550a2dfde7d3711cc605ff5cb8be9bf39f7/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/04 20:52:47 Job state: DONE
22/09/04 20:52:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd_44e33d5f-c471-462e-8bd1-3e8616cedf6b
22/09/04 20:52:47 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0600-jenkins-0904205238-52390abd_44e33d5f-c471-462e-8bd1-3e8616cedf6b
--- PASS: TestEmitParDoAfterGBK (10.15s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 112.787s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 21s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/4qjob5gyusurm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6553
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6553/display/redirect>
Changes:
------------------------------------------
[...truncated 2.51 MB...]
22/09/04 14:52:35 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45101 for data ...
22/09/04 14:52:35 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 14:52:35 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 14:52:35 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 14:52:35 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 14:52:35 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ecd26a0d786b66af8db5556596f58c1d22260b366f6de0ccd23797db67752248. Logs:
2022/09/04 14:52:35 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-904"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:40817"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test2733"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test2733"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42941"} artifact_endpoint:{url:"localhost:33023"} control_endpoint:{url:"localhost:36579"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 14:52:35 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:46669
2022/09/04 14:52:35 Downloaded: /tmp/staged/1-worker-13-1662303150662443047 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 14:52:35 control response channel closed
22/09/04 14:52:36 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 14:52:36 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 14:52:36 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 14:52:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/04 14:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46469 for control ...
22/09/04 14:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43055 for logging ...
22/09/04 14:52:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/04 14:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45399 for data ...
22/09/04 14:52:36 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 14:52:36 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 14:52:36 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 14:52:36 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 14:52:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 92ac1eb4aa4b3f2c71516334f48af7f231c21a65cd373cefb9ebfd60d6177ecd. Logs:
2022/09/04 14:52:36 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-904"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:40817"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test2733"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test2733"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43055"} artifact_endpoint:{url:"localhost:36077"} control_endpoint:{url:"localhost:46469"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 14:52:36 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:36183
2022/09/04 14:52:36 Downloaded: /tmp/staged/1-worker-13-1662303150662443047 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 14:52:36 control response channel closed
22/09/04 14:52:37 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 14:52:38 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 14:52:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/04 14:52:38 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 14:52:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34089 for control ...
22/09/04 14:52:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38587 for logging ...
22/09/04 14:52:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/04 14:52:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35799 for data ...
22/09/04 14:52:38 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 14:52:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41681 for state ...
22/09/04 14:52:38 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 14:52:38 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 14:52:38 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 14:52:38 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container fd0df1f4d0f07b59324542d0449e0ca706375d034d243a3ddd0511e78036b164. Logs:
2022/09/04 14:52:38 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-904"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:40817"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test2733"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test2733"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38587"} artifact_endpoint:{url:"localhost:35321"} control_endpoint:{url:"localhost:34089"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 14:52:38 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:46845
2022/09/04 14:52:38 Downloaded: /tmp/staged/1-worker-13-1662303150662443047 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/04 14:52:38 control response channel closed
Worker exited successfully!
22/09/04 14:52:39 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 14:52:39 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 14:52:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042_7d6d3203-281b-4c72-82c1-0d9868a833ba: Pipeline translated successfully. Computing outputs
22/09/04 14:52:39 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 14:52:39 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 14:52:40 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 14:52:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/04 14:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44251 for control ...
22/09/04 14:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36171 for logging ...
22/09/04 14:52:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/04 14:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38075 for data ...
22/09/04 14:52:40 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 14:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39533 for state ...
22/09/04 14:52:40 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 14:52:40 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 14:52:40 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 14:52:40 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9051d1b607121feb8e59e0b3f6e746dcc0fc5d1f9009a11e77091bf65dd0dcac. Logs:
2022/09/04 14:52:39 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-904"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:40817"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test2733"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test2733"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36171"} artifact_endpoint:{url:"localhost:35993"} control_endpoint:{url:"localhost:44251"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/1-go-/tmp/worker-13-1662303150662443047" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 14:52:39 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:44293
2022/09/04 14:52:40 Downloaded: /tmp/staged/1-worker-13-1662303150662443047 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 14:52:40 control response channel closed
22/09/04 14:52:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042_7d6d3203-281b-4c72-82c1-0d9868a833ba finished.
22/09/04 14:52:40 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-904_58e16fe4-b9ce-43f3-ae84-fad0efcb00c7.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/142c63bb9fd04360b502bb7437e8eb7e5ddde5a270242c794f8e1764f91e4562/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/04 14:52:40 Job state: DONE
22/09/04 14:52:40 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042_7d6d3203-281b-4c72-82c1-0d9868a833ba
22/09/04 14:52:40 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0904-jenkins-0904145232-2e9d6042_7d6d3203-281b-4c72-82c1-0d9868a833ba
--- PASS: TestEmitParDoAfterGBK (10.16s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 122.971s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 17s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/vnallzm2pvc6k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6552
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6552/display/redirect>
Changes:
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 08:53:02 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 08:53:02 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 08:53:02 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 697e4211f32c44b2dbb48b67e43ef6d95c24634c65a5713cdc5d73ba9072d437. Logs:
2022/09/04 08:53:01 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-806"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:36483"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test8320"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test8320"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33257"} artifact_endpoint:{url:"localhost:41445"} control_endpoint:{url:"localhost:40213"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 08:53:01 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:39221
2022/09/04 08:53:02 Downloaded: /tmp/staged/1-worker-13-1662281577407347690 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 08:53:02 control response channel closed
22/09/04 08:53:02 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 08:53:03 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 08:53:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/04 08:53:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 08:53:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36501 for control ...
22/09/04 08:53:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40807 for logging ...
22/09/04 08:53:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/04 08:53:03 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43879 for data ...
22/09/04 08:53:03 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 08:53:03 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 08:53:03 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 08:53:03 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 08:53:03 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 16de284e2f2e49f7e40d1f147e3bb709fd5765f0e81f5538f16150d8ec5d25c2. Logs:
2022/09/04 08:53:03 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-806"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:36483"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test8320"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test8320"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40807"} artifact_endpoint:{url:"localhost:34553"} control_endpoint:{url:"localhost:36501"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 08:53:03 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:41177
2022/09/04 08:53:03 Downloaded: /tmp/staged/1-worker-13-1662281577407347690 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 08:53:03 control response channel closed
22/09/04 08:53:04 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 08:53:04 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 08:53:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 08:53:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/04 08:53:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46139 for control ...
22/09/04 08:53:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35681 for logging ...
22/09/04 08:53:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/04 08:53:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45355 for data ...
22/09/04 08:53:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 08:53:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41941 for state ...
22/09/04 08:53:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 08:53:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 08:53:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 08:53:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ded3d33127f97857d5729b444b2e02915d4b500cec8ce143d27805997d297d41. Logs:
2022/09/04 08:53:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-806"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:36483"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test8320"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test8320"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35681"} artifact_endpoint:{url:"localhost:35535"} control_endpoint:{url:"localhost:46139"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 08:53:05 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:39021
2022/09/04 08:53:05 Downloaded: /tmp/staged/1-worker-13-1662281577407347690 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 08:53:05 control response channel closed
22/09/04 08:53:05 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 08:53:05 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 08:53:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc_3eafc970-3812-4b4b-9f98-a48d3f7f8430: Pipeline translated successfully. Computing outputs
22/09/04 08:53:05 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 08:53:06 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 08:53:06 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 08:53:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/04 08:53:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37219 for control ...
22/09/04 08:53:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41459 for logging ...
22/09/04 08:53:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/04 08:53:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45637 for data ...
22/09/04 08:53:06 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 08:53:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36327 for state ...
22/09/04 08:53:06 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 08:53:06 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 08:53:06 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 08:53:06 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container bd6268644265acc6ef65fe1df5f23c3678c468e4a85c1a748cf2820ae766e48c. Logs:
2022/09/04 08:53:06 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-806"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:36483"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test8320"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test8320"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41459"} artifact_endpoint:{url:"localhost:46287"} control_endpoint:{url:"localhost:37219"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/1-go-/tmp/worker-13-1662281577407347690" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 08:53:06 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:41993
2022/09/04 08:53:06 Downloaded: /tmp/staged/1-worker-13-1662281577407347690 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 08:53:06 control response channel closed
22/09/04 08:53:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc_3eafc970-3812-4b4b-9f98-a48d3f7f8430 finished.
22/09/04 08:53:07 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-806_ba55e818-658e-43f6-a210-25f086484a77.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/1cc047fa518d450835b4a7edfd02b02c4eb871e80c86d176ab893c0438218901/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/04 08:53:07 Job state: DONE
22/09/04 08:53:07 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc_3eafc970-3812-4b4b-9f98-a48d3f7f8430
22/09/04 08:53:07 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0806-jenkins-0904085259-2388bfdc_3eafc970-3812-4b4b-9f98-a48d3f7f8430
--- PASS: TestEmitParDoAfterGBK (10.15s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 113.664s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 41s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/6surwr7zfkhdo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6551
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6551/display/redirect?page=changes>
Changes:
[noreply] [#19857] Migrate to using a memory aware cache within the Python SDK
------------------------------------------
[...truncated 2.52 MB...]
22/09/04 02:52:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38029 for data ...
22/09/04 02:52:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 02:52:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 02:52:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 02:52:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 02:52:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 3559eb64c3a0608fea66e3384df8d6bca989c73229dcbeb4191a80d64ac2179d. Logs:
2022/09/04 02:52:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-751"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38381"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test21283"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test21283"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34345"} artifact_endpoint:{url:"localhost:37023"} control_endpoint:{url:"localhost:33127"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 02:52:05 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:33549
2022/09/04 02:52:05 Downloaded: /tmp/staged/1-worker-13-1662259920641756138 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 02:52:05 control response channel closed
22/09/04 02:52:06 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 02:52:06 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 02:52:06 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 02:52:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/04 02:52:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36933 for control ...
22/09/04 02:52:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35943 for logging ...
22/09/04 02:52:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/04 02:52:06 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40531 for data ...
22/09/04 02:52:06 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 02:52:07 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 02:52:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 02:52:07 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 02:52:07 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 12a4f8ce46cc7659faf11869d1591c24388b9ec2c8f44b2ed5c40bd53c9a5f4b. Logs:
2022/09/04 02:52:06 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-751"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38381"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test21283"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test21283"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35943"} artifact_endpoint:{url:"localhost:44469"} control_endpoint:{url:"localhost:36933"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 02:52:06 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:43937
2022/09/04 02:52:06 Downloaded: /tmp/staged/1-worker-13-1662259920641756138 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/04 02:52:07 control response channel closed
Worker exited successfully!
22/09/04 02:52:07 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 02:52:08 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 02:52:08 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 02:52:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/04 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41099 for control ...
22/09/04 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39301 for logging ...
22/09/04 02:52:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/04 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45315 for data ...
22/09/04 02:52:08 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34769 for state ...
22/09/04 02:52:08 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 02:52:08 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 02:52:08 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 02:52:08 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 52ed6a6f0286ddf2758be99f40167a7d44687593ad75cbc4125d14b380374985. Logs:
2022/09/04 02:52:08 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-751"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38381"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test21283"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test21283"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39301"} artifact_endpoint:{url:"localhost:35571"} control_endpoint:{url:"localhost:41099"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 02:52:08 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:35443
2022/09/04 02:52:08 Downloaded: /tmp/staged/1-worker-13-1662259920641756138 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/04 02:52:08 control response channel closed
Worker exited successfully!
22/09/04 02:52:09 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 02:52:09 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 02:52:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392_1c854c34-fde3-40ab-b138-7757074bd0fb: Pipeline translated successfully. Computing outputs
22/09/04 02:52:09 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/04 02:52:09 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/04 02:52:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/04 02:52:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/04 02:52:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33847 for control ...
22/09/04 02:52:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42127 for logging ...
22/09/04 02:52:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/04 02:52:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45117 for data ...
22/09/04 02:52:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/04 02:52:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37557 for state ...
22/09/04 02:52:10 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/04 02:52:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/04 02:52:10 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/04 02:52:10 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container acce9a7cb24d3a04b268e64b74abf4c80a3de57308eabe75f863f5a84635ba8f. Logs:
2022/09/04 02:52:10 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-751"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38381"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test21283"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test21283"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42127"} artifact_endpoint:{url:"localhost:45125"} control_endpoint:{url:"localhost:33847"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/1-go-/tmp/worker-13-1662259920641756138" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/04 02:52:10 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:44589
2022/09/04 02:52:10 Downloaded: /tmp/staged/1-worker-13-1662259920641756138 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/04 02:52:10 control response channel closed
22/09/04 02:52:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392_1c854c34-fde3-40ab-b138-7757074bd0fb finished.
22/09/04 02:52:10 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-751_574447c4-51f0-41e2-af0c-ce229a737e78.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/f94c1289ec48b727787afd09ab1e6480e8561cdcdc86b251e8a419ad59dbf137/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/04 02:52:10 Job state: DONE
22/09/04 02:52:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392_1c854c34-fde3-40ab-b138-7757074bd0fb
22/09/04 02:52:10 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0751-jenkins-0904025202-bbe3f392_1c854c34-fde3-40ab-b138-7757074bd0fb
--- PASS: TestEmitParDoAfterGBK (10.23s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 113.678s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 12m 44s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/5k73qtkirrfau
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6550
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6550/display/redirect>
Changes:
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 20:52:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 20:52:18 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 20:52:18 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 67b0844e90659e83e1f6d0a30aa425be0b27a41af613590fe8fea157d8273c02. Logs:
2022/09/03 20:52:18 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-9"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57609"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test32331"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test32331"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk09-jenkins-0903205215-d985aea"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36711"} artifact_endpoint:{url:"localhost:43871"} control_endpoint:{url:"localhost:33193"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 20:52:18 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:42057
2022/09/03 20:52:18 Downloaded: /tmp/staged/1-worker-13-1662238333565153559 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/03 20:52:18 control response channel closed
Worker exited successfully!
22/09/03 20:52:18 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 20:52:19 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 20:52:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 20:52:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/03 20:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40363 for control ...
22/09/03 20:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43139 for logging ...
22/09/03 20:52:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/03 20:52:19 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34533 for data ...
22/09/03 20:52:19 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 20:52:19 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 20:52:19 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 20:52:19 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 20:52:19 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ccc292edc4b3650573e42484102b03ad970ca03a6531cd1732c259e9d1fe1a2a. Logs:
2022/09/03 20:52:19 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-9"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57609"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test32331"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test32331"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk09-jenkins-0903205215-d985aea"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43139"} artifact_endpoint:{url:"localhost:37603"} control_endpoint:{url:"localhost:40363"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 20:52:19 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:34959
2022/09/03 20:52:19 Downloaded: /tmp/staged/1-worker-13-1662238333565153559 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 20:52:19 control response channel closed
22/09/03 20:52:20 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 20:52:21 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 20:52:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 20:52:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/03 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39013 for control ...
22/09/03 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33447 for logging ...
22/09/03 20:52:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/03 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43809 for data ...
22/09/03 20:52:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 20:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36407 for state ...
22/09/03 20:52:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 20:52:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 20:52:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 20:52:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 36e975021fabc7ba9876897fb0b9bd2e92cc639bfa358e278eefd6e2620cb01b. Logs:
2022/09/03 20:52:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-9"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57609"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test32331"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test32331"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk09-jenkins-0903205215-d985aea"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33447"} artifact_endpoint:{url:"localhost:45449"} control_endpoint:{url:"localhost:39013"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 20:52:21 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:41239
2022/09/03 20:52:21 Downloaded: /tmp/staged/1-worker-13-1662238333565153559 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 20:52:21 control response channel closed
22/09/03 20:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 20:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 20:52:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk09-jenkins-0903205215-d985aea_0725dc48-3170-4c83-b551-61ee7228a2af: Pipeline translated successfully. Computing outputs
22/09/03 20:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 20:52:22 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 20:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/03 20:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33543 for control ...
22/09/03 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39795 for logging ...
22/09/03 20:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/03 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42083 for data ...
22/09/03 20:52:23 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 20:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36823 for state ...
22/09/03 20:52:23 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 20:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 20:52:23 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 20:52:23 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e9cb5cefb0269c4fb6ee1d71f903bfe2f36cd2f2abbc55fbb30e3b2bb7f54af2. Logs:
2022/09/03 20:52:22 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-9"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57609"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test32331"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test32331"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk09-jenkins-0903205215-d985aea"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39795"} artifact_endpoint:{url:"localhost:40301"} control_endpoint:{url:"localhost:33543"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/1-go-/tmp/worker-13-1662238333565153559" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 20:52:22 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:38471
2022/09/03 20:52:23 Downloaded: /tmp/staged/1-worker-13-1662238333565153559 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/03 20:52:23 control response channel closed
Worker exited successfully!
22/09/03 20:52:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk09-jenkins-0903205215-d985aea_0725dc48-3170-4c83-b551-61ee7228a2af finished.
22/09/03 20:52:23 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-9_257957a4-c7b7-4fa4-9173-f5ec235ce13d.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/0865f07e3a6ddfab656be85c112589672d8acc1b761305d8deeafdfa16c4a216/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/03 20:52:23 Job state: DONE
22/09/03 20:52:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk09-jenkins-0903205215-d985aea_0725dc48-3170-4c83-b551-61ee7228a2af
22/09/03 20:52:23 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk09-jenkins-0903205215-d985aea_0725dc48-3170-4c83-b551-61ee7228a2af
--- PASS: TestEmitParDoAfterGBK (10.23s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 114.034s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 12m 58s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/5draruuplbs4c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6549
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6549/display/redirect>
Changes:
------------------------------------------
[...truncated 2.51 MB...]
22/09/03 14:53:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35879 for data ...
22/09/03 14:53:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 14:53:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 14:53:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 14:53:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 14:53:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 63b8a7cf4bf119b6137b8a5a11941a523a7ca161a84ef470412dd5f2389823c4. Logs:
2022/09/03 14:53:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-4"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57459"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test13988"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test13988"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44279"} artifact_endpoint:{url:"localhost:43567"} control_endpoint:{url:"localhost:43993"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 14:53:05 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:39807
2022/09/03 14:53:05 Downloaded: /tmp/staged/1-worker-13-1662216780759042468 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 14:53:05 control response channel closed
22/09/03 14:53:06 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 14:53:06 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 14:53:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 14:53:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/03 14:53:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46223 for control ...
22/09/03 14:53:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44495 for logging ...
22/09/03 14:53:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/03 14:53:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37243 for data ...
22/09/03 14:53:07 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 14:53:07 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 14:53:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 14:53:07 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 14:53:07 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 457f4c801bfa4d83220f06a7959a8297a55b4fd6ef34a948cce603afd4c46037. Logs:
2022/09/03 14:53:06 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-4"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57459"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test13988"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test13988"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44495"} artifact_endpoint:{url:"localhost:35529"} control_endpoint:{url:"localhost:46223"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 14:53:06 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:34807
2022/09/03 14:53:07 Downloaded: /tmp/staged/1-worker-13-1662216780759042468 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 14:53:07 control response channel closed
22/09/03 14:53:07 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 14:53:08 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 14:53:08 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 14:53:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/03 14:53:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43433 for control ...
22/09/03 14:53:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41449 for logging ...
22/09/03 14:53:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/03 14:53:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36577 for data ...
22/09/03 14:53:08 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 14:53:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43147 for state ...
22/09/03 14:53:08 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 14:53:08 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 14:53:08 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 14:53:08 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e0ee7706ae000dea445c8a3dda4fe64d42ab483a86bb2292e07f4be719c97007. Logs:
2022/09/03 14:53:08 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-4"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57459"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test13988"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test13988"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41449"} artifact_endpoint:{url:"localhost:39775"} control_endpoint:{url:"localhost:43433"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 14:53:08 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:37415
2022/09/03 14:53:08 Downloaded: /tmp/staged/1-worker-13-1662216780759042468 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/03 14:53:08 control response channel closed
Worker exited successfully!
22/09/03 14:53:09 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 14:53:09 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 14:53:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6_aecf34e6-f720-4fef-8292-b021d66cf3a7: Pipeline translated successfully. Computing outputs
22/09/03 14:53:09 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 14:53:10 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 14:53:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 14:53:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/03 14:53:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46835 for control ...
22/09/03 14:53:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39181 for logging ...
22/09/03 14:53:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/03 14:53:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40533 for data ...
22/09/03 14:53:10 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 14:53:10 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43421 for state ...
22/09/03 14:53:10 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 14:53:10 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 14:53:10 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 14:53:10 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1a7d1a23e7c523b2552e99010b312e058057d9a4548473387ccec9b98542d5b9. Logs:
2022/09/03 14:53:10 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-4"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:57459"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test13988"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test13988"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39181"} artifact_endpoint:{url:"localhost:45499"} control_endpoint:{url:"localhost:46835"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/1-go-/tmp/worker-13-1662216780759042468" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 14:53:10 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:37549
2022/09/03 14:53:10 Downloaded: /tmp/staged/1-worker-13-1662216780759042468 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/03 14:53:10 control response channel closed
Worker exited successfully!
22/09/03 14:53:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6_aecf34e6-f720-4fef-8292-b021d66cf3a7 finished.
22/09/03 14:53:11 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-4_59dedf69-12ea-476b-825f-9814b98694ad.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/dbdf611824948bdc49bfd834c36759df1c82333b002c16ff635f54c16e889e62/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/03 14:53:11 Job state: DONE
22/09/03 14:53:11 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6_aecf34e6-f720-4fef-8292-b021d66cf3a7
22/09/03 14:53:11 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk04-jenkins-0903145302-4ccc2cd6_aecf34e6-f720-4fef-8292-b021d66cf3a7
--- PASS: TestEmitParDoAfterGBK (10.53s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 112.266s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 45s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/ubzxugc5herie
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6548
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6548/display/redirect>
Changes:
------------------------------------------
[...truncated 2.51 MB...]
22/09/03 08:52:36 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38569 for data ...
22/09/03 08:52:36 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 08:52:36 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 08:52:36 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 08:52:36 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 08:52:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 92bf661b43c7b9774162ce7782805218d4df72943b95f3e9fc0e8e68f8e90aea. Logs:
2022/09/03 08:52:36 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-630"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51219"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1817"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1817"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42167"} artifact_endpoint:{url:"localhost:46881"} control_endpoint:{url:"localhost:46005"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 08:52:36 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:34067
2022/09/03 08:52:36 Downloaded: /tmp/staged/1-worker-13-1662195152222474555 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 08:52:36 control response channel closed
22/09/03 08:52:37 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 08:52:38 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 08:52:38 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 08:52:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/03 08:52:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44149 for control ...
22/09/03 08:52:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36409 for logging ...
22/09/03 08:52:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/03 08:52:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41323 for data ...
22/09/03 08:52:38 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 08:52:38 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 08:52:38 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 08:52:38 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 08:52:38 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e563286c324e599e2225e03acee2dd613a00b9d174c424d92768d7bcad79814b. Logs:
2022/09/03 08:52:38 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-630"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51219"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1817"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1817"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36409"} artifact_endpoint:{url:"localhost:35359"} control_endpoint:{url:"localhost:44149"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 08:52:38 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:40617
2022/09/03 08:52:38 Downloaded: /tmp/staged/1-worker-13-1662195152222474555 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 08:52:38 control response channel closed
22/09/03 08:52:39 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 08:52:39 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 08:52:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/03 08:52:40 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 08:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42019 for control ...
22/09/03 08:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38939 for logging ...
22/09/03 08:52:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/03 08:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43927 for data ...
22/09/03 08:52:40 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 08:52:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46851 for state ...
22/09/03 08:52:40 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 08:52:40 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 08:52:40 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 08:52:40 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c1876ce37841c7d614990a0f32d7e9837363731ca4d94525b89bba951fde7163. Logs:
2022/09/03 08:52:39 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-630"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51219"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1817"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1817"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38939"} artifact_endpoint:{url:"localhost:44001"} control_endpoint:{url:"localhost:42019"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 08:52:39 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:42681
2022/09/03 08:52:40 Downloaded: /tmp/staged/1-worker-13-1662195152222474555 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/03 08:52:40 control response channel closed
Worker exited successfully!
22/09/03 08:52:40 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 08:52:40 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 08:52:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b_3e80f436-f178-4035-a37a-f768d78fd6c0: Pipeline translated successfully. Computing outputs
22/09/03 08:52:40 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 08:52:41 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 08:52:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 08:52:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/03 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33683 for control ...
22/09/03 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45605 for logging ...
22/09/03 08:52:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/03 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34271 for data ...
22/09/03 08:52:41 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33313 for state ...
22/09/03 08:52:41 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 08:52:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 08:52:41 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 08:52:41 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 298a50912911db730a86409e5889a642bc1255a2e64d05978610a934446550e8. Logs:
2022/09/03 08:52:41 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-630"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51219"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test1817"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test1817"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45605"} artifact_endpoint:{url:"localhost:40641"} control_endpoint:{url:"localhost:33683"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/1-go-/tmp/worker-13-1662195152222474555" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 08:52:41 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:45763
2022/09/03 08:52:41 Downloaded: /tmp/staged/1-worker-13-1662195152222474555 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 08:52:41 control response channel closed
22/09/03 08:52:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b_3e80f436-f178-4035-a37a-f768d78fd6c0 finished.
22/09/03 08:52:42 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-630_17f1d6ba-f561-4265-99a9-bfbf31639475.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/7f22879a0809c5bb91fad175c4e948f4191d35cefdb1e54c120dbcaa3b339e47/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/03 08:52:42 Job state: DONE
22/09/03 08:52:42 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b_3e80f436-f178-4035-a37a-f768d78fd6c0
22/09/03 08:52:42 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0630-jenkins-0903085234-13ea6b7b_3e80f436-f178-4035-a37a-f768d78fd6c0
--- PASS: TestEmitParDoAfterGBK (10.15s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 111.349s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 13s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/sltnc6gxguxae
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6547/display/redirect?page=changes>
Changes:
[noreply] [Playground] [Infrastructure] Deleting the Cloud Storage Client (#22722)
[noreply] Updates Java RunInference to infer Python dependencies when possible
[noreply] Adding TensorFlow support to the Machine Learning overview page (#22949)
------------------------------------------
[...truncated 2.52 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 02:52:04 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 02:52:04 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 02:52:04 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 118db606465c943589c9b23ad268e34b95ca196ac71fbf2eeaeac447090344d7. Logs:
2022/09/03 02:52:03 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-533"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38569"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18308"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18308"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34155"} artifact_endpoint:{url:"localhost:35271"} control_endpoint:{url:"localhost:36115"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 02:52:03 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:38359
2022/09/03 02:52:04 Downloaded: /tmp/staged/1-worker-13-1662173519362990749 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 02:52:04 control response channel closed
22/09/03 02:52:04 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 02:52:05 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 02:52:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 02:52:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/03 02:52:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46045 for control ...
22/09/03 02:52:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44153 for logging ...
22/09/03 02:52:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/03 02:52:05 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42573 for data ...
22/09/03 02:52:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 02:52:05 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 02:52:05 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 02:52:05 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 02:52:05 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ecf731f9dbb68843e47fd143c996f330c1ccc4dac6f7c0cd481d4c5c0664d663. Logs:
2022/09/03 02:52:05 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-533"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38569"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18308"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18308"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44153"} artifact_endpoint:{url:"localhost:34969"} control_endpoint:{url:"localhost:46045"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 02:52:05 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:45417
2022/09/03 02:52:05 Downloaded: /tmp/staged/1-worker-13-1662173519362990749 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 02:52:05 control response channel closed
22/09/03 02:52:06 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 02:52:06 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 02:52:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 02:52:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/03 02:52:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43851 for control ...
22/09/03 02:52:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45921 for logging ...
22/09/03 02:52:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/03 02:52:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44383 for data ...
22/09/03 02:52:07 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 02:52:07 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35377 for state ...
22/09/03 02:52:07 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 02:52:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 02:52:07 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 02:52:07 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d1877b97ed2006ac2919a17d18144624b8255880c7ebb7901cdf5ffcfe6ac553. Logs:
2022/09/03 02:52:06 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-533"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38569"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18308"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18308"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45921"} artifact_endpoint:{url:"localhost:41271"} control_endpoint:{url:"localhost:43851"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 02:52:06 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:38921
2022/09/03 02:52:07 Downloaded: /tmp/staged/1-worker-13-1662173519362990749 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 02:52:07 control response channel closed
22/09/03 02:52:07 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 02:52:07 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 02:52:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216_a487e295-a187-43e7-b228-e4d64370313e: Pipeline translated successfully. Computing outputs
22/09/03 02:52:07 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/03 02:52:08 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/03 02:52:08 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/03 02:52:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/03 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33571 for control ...
22/09/03 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46525 for logging ...
22/09/03 02:52:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/03 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38563 for data ...
22/09/03 02:52:08 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/03 02:52:08 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44695 for state ...
22/09/03 02:52:08 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/03 02:52:08 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/03 02:52:08 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/03 02:52:08 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 318a8abf591ab8802195b8e2fbd5b59014c8eab5c345cc66289850ed0599eddf. Logs:
2022/09/03 02:52:08 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-533"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38569"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18308"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18308"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:46525"} artifact_endpoint:{url:"localhost:41933"} control_endpoint:{url:"localhost:33571"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/1-go-/tmp/worker-13-1662173519362990749" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/03 02:52:08 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:38597
2022/09/03 02:52:08 Downloaded: /tmp/staged/1-worker-13-1662173519362990749 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/03 02:52:08 control response channel closed
22/09/03 02:52:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216_a487e295-a187-43e7-b228-e4d64370313e finished.
22/09/03 02:52:09 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-533_ede7d98e-9f58-4cc6-b74b-41a3ecc7c4eb.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/ba10496f14d7f478fc0e6fdd105e1d20e7bee5a74700790f9b4d8589851fc136/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/03 02:52:09 Job state: DONE
22/09/03 02:52:09 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216_a487e295-a187-43e7-b228-e4d64370313e
22/09/03 02:52:09 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0533-jenkins-0903025201-d5c98216_a487e295-a187-43e7-b228-e4d64370313e
--- PASS: TestEmitParDoAfterGBK (10.11s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 111.781s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 12m 42s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/rwz6wjhmc3rdk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6546
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6546/display/redirect?page=changes>
Changes:
[noreply] Bump cloud.google.com/go/storage from 1.25.0 to 1.26.0 in /sdks (#22954)
[noreply] [BEAM-22859] Allow the specification of extra packages for external
[noreply] [Tour of Beam]: Welcome Screen frontend layout (#22794)
[noreply] Remove redundant testEventTimeTimerSetWithinAllowedLateness sickbay
[noreply] Adding support for Beam Schema Rows with BQ DIRECT_READ (#22926)
[noreply] Add java Bigquery IO known issue to beam 2.40 release blogpost (#22611)
[noreply] Update playground_deploy_examples.yml
[noreply] Add run-inference component for autolabeling (#22971)
------------------------------------------
[...truncated 2.52 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 20:53:24 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 20:53:24 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 20:53:24 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 682a1a0b8416e37c8034e0bd8a6cf27c97d1d4ecb63c1419b68e02051f1d5ee3. Logs:
2022/09/02 20:53:24 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-684"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55375"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test23689"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test23689"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43981"} artifact_endpoint:{url:"localhost:41447"} control_endpoint:{url:"localhost:36903"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 20:53:24 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:38983
2022/09/02 20:53:24 Downloaded: /tmp/staged/1-worker-13-1662151999670798319 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 20:53:24 control response channel closed
22/09/02 20:53:25 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 20:53:25 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 20:53:26 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 20:53:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/02 20:53:26 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33683 for control ...
22/09/02 20:53:26 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45629 for logging ...
22/09/02 20:53:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/02 20:53:26 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34361 for data ...
22/09/02 20:53:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 20:53:26 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 20:53:26 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 20:53:26 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 20:53:26 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 4c10ed7e2c031ef0a5310c3a2547cf9d84edf46899626466007f016e75fcd142. Logs:
2022/09/02 20:53:25 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-684"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55375"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test23689"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test23689"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45629"} artifact_endpoint:{url:"localhost:35405"} control_endpoint:{url:"localhost:33683"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 20:53:25 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:35701
2022/09/02 20:53:26 Downloaded: /tmp/staged/1-worker-13-1662151999670798319 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 20:53:26 control response channel closed
22/09/02 20:53:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 20:53:27 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 20:53:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/02 20:53:27 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 20:53:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45309 for control ...
22/09/02 20:53:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36129 for logging ...
22/09/02 20:53:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/02 20:53:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45533 for data ...
22/09/02 20:53:27 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 20:53:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34617 for state ...
22/09/02 20:53:27 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 20:53:27 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 20:53:27 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 20:53:27 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container ed2b0e8439b7d0bfed588e8a0e35e702d41680c919ef3e384f1aa5039c38d176. Logs:
2022/09/02 20:53:27 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-684"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55375"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test23689"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test23689"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36129"} artifact_endpoint:{url:"localhost:35039"} control_endpoint:{url:"localhost:45309"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 20:53:27 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:36569
2022/09/02 20:53:27 Downloaded: /tmp/staged/1-worker-13-1662151999670798319 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/02 20:53:27 control response channel closed
Worker exited successfully!
22/09/02 20:53:28 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 20:53:28 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 20:53:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba_0ce26b87-17a7-42dd-90eb-42e44d08c3bb: Pipeline translated successfully. Computing outputs
22/09/02 20:53:28 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 20:53:28 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 20:53:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 20:53:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/02 20:53:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46565 for control ...
22/09/02 20:53:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45919 for logging ...
22/09/02 20:53:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/02 20:53:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38031 for data ...
22/09/02 20:53:29 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 20:53:29 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46593 for state ...
22/09/02 20:53:29 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 20:53:29 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 20:53:29 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 20:53:29 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 3c5c0236d75fd99823eb26f4bdf107477cfc01c0417926bd5bc2a627a9b641db. Logs:
2022/09/02 20:53:29 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-684"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55375"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test23689"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test23689"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45919"} artifact_endpoint:{url:"localhost:42957"} control_endpoint:{url:"localhost:46565"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/1-go-/tmp/worker-13-1662151999670798319" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 20:53:29 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:39661
2022/09/02 20:53:29 Downloaded: /tmp/staged/1-worker-13-1662151999670798319 (sha256: 32e87e6cd109bdea60539e1871231f60ab636c983a896ecdd7a6f12d232cc887, size: 26517776)
Error Setting Rlimit operation not permitted
2022/09/02 20:53:29 control response channel closed
Worker exited successfully!
22/09/02 20:53:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba_0ce26b87-17a7-42dd-90eb-42e44d08c3bb finished.
22/09/02 20:53:30 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-684_487cd392-9bbd-43f4-a225-2d87ecf04f4b.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/ce2d82a7558687c23ef5c2c9fa2059b2cce703d13673a49825f8ab59481e61d9/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/02 20:53:30 Job state: DONE
22/09/02 20:53:30 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba_0ce26b87-17a7-42dd-90eb-42e44d08c3bb
22/09/02 20:53:30 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0684-jenkins-0902205321-ce387fba_0ce26b87-17a7-42dd-90eb-42e44d08c3bb
--- PASS: TestEmitParDoAfterGBK (10.38s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 122.475s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 14m 2s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/pkp73kfqirxka
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6545
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6545/display/redirect?page=changes>
Changes:
[noreply] [Tour Of Beam][backend] get unit content (#22967)
[noreply] Allows to use databaseio with postgres driver (#22941)
------------------------------------------
[...truncated 2.52 MB...]
22/09/02 14:52:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44695 for data ...
22/09/02 14:52:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 14:52:49 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 14:52:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 14:52:49 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 14:52:49 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container bbb97c68b6e5298aa19facd477429d696f950ab6702a1566cf435e349754ffc0. Logs:
2022/09/02 14:52:49 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-116"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:37325"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7129"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7129"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35525"} artifact_endpoint:{url:"localhost:35765"} control_endpoint:{url:"localhost:39385"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 14:52:49 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:38777
2022/09/02 14:52:49 Downloaded: /tmp/staged/1-worker-13-1662130364710011716 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 14:52:49 control response channel closed
22/09/02 14:52:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 14:52:50 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 14:52:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 14:52:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/02 14:52:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37905 for control ...
22/09/02 14:52:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43677 for logging ...
22/09/02 14:52:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/02 14:52:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44967 for data ...
22/09/02 14:52:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 14:52:51 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 14:52:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 14:52:51 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 14:52:51 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 8bfd3e1973659222a160cfaaa007fe60cdfce8a69476dc21a55a107d59a140c9. Logs:
2022/09/02 14:52:50 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-116"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:37325"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7129"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7129"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43677"} artifact_endpoint:{url:"localhost:36959"} control_endpoint:{url:"localhost:37905"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 14:52:50 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:36009
2022/09/02 14:52:51 Downloaded: /tmp/staged/1-worker-13-1662130364710011716 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 14:52:51 control response channel closed
22/09/02 14:52:51 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 14:52:52 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 14:52:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/02 14:52:52 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 14:52:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43381 for control ...
22/09/02 14:52:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43683 for logging ...
22/09/02 14:52:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/02 14:52:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46257 for data ...
22/09/02 14:52:52 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 14:52:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46697 for state ...
22/09/02 14:52:52 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 14:52:52 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 14:52:52 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 14:52:52 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container fd437fc28f80619eff7a9a68dcff561d3f09a6d19482bed4e0505a34375dbd43. Logs:
2022/09/02 14:52:52 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-116"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:37325"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7129"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7129"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43683"} artifact_endpoint:{url:"localhost:35473"} control_endpoint:{url:"localhost:43381"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 14:52:52 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:38609
2022/09/02 14:52:52 Downloaded: /tmp/staged/1-worker-13-1662130364710011716 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 14:52:52 control response channel closed
22/09/02 14:52:53 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 14:52:53 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 14:52:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38_70c21cda-a76e-4b33-be6b-bfad0bf0ddaf: Pipeline translated successfully. Computing outputs
22/09/02 14:52:53 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 14:52:53 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 14:52:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/02 14:52:54 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 14:52:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46275 for control ...
22/09/02 14:52:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45683 for logging ...
22/09/02 14:52:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/02 14:52:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44803 for data ...
22/09/02 14:52:54 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 14:52:54 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45785 for state ...
22/09/02 14:52:54 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 14:52:54 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 14:52:54 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 14:52:54 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d2390480f1c8df0f06bd69ac4f6e0d9bf814d004177e4e59730336405aa05e10. Logs:
2022/09/02 14:52:54 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-116"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:37325"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test7129"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test7129"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45683"} artifact_endpoint:{url:"localhost:42901"} control_endpoint:{url:"localhost:46275"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/1-go-/tmp/worker-13-1662130364710011716" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 14:52:54 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:38341
2022/09/02 14:52:54 Downloaded: /tmp/staged/1-worker-13-1662130364710011716 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 14:52:54 control response channel closed
22/09/02 14:52:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38_70c21cda-a76e-4b33-be6b-bfad0bf0ddaf finished.
22/09/02 14:52:55 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-116_d73b4063-b5c3-4ab3-880f-de05f0542eed.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/e8ac14fc98992966009dfea60039cad85efb0da9c175fed340f734639cf60f23/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/02 14:52:55 Job state: DONE
22/09/02 14:52:55 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38_70c21cda-a76e-4b33-be6b-bfad0bf0ddaf
22/09/02 14:52:55 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0116-jenkins-0902145246-36db3e38_70c21cda-a76e-4b33-be6b-bfad0bf0ddaf
--- PASS: TestEmitParDoAfterGBK (10.37s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 116.393s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 29s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/b4fz75mkznsfy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6544
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6544/display/redirect>
Changes:
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 08:52:37 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 08:52:37 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 08:52:37 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d1f92497034ea9f25d6775c0d6284f0847197b76cf3dec6a2c6fb490160f8d9b. Logs:
2022/09/02 08:52:37 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-232"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:35977"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4014"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4014"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45679"} artifact_endpoint:{url:"localhost:44969"} control_endpoint:{url:"localhost:39571"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 08:52:37 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:33733
2022/09/02 08:52:37 Downloaded: /tmp/staged/1-worker-13-1662108752054159145 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 08:52:37 control response channel closed
22/09/02 08:52:38 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 08:52:38 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 08:52:39 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 08:52:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/02 08:52:39 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40515 for control ...
22/09/02 08:52:39 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40735 for logging ...
22/09/02 08:52:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/02 08:52:39 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46525 for data ...
22/09/02 08:52:39 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 08:52:39 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 08:52:39 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 08:52:39 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 08:52:39 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b8b7eb2d690a6c8897dc20f57aeaf36bbced198870f37efcafd917cb075680dd. Logs:
2022/09/02 08:52:39 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-232"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:35977"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4014"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4014"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40735"} artifact_endpoint:{url:"localhost:36161"} control_endpoint:{url:"localhost:40515"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 08:52:39 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:39925
2022/09/02 08:52:39 Downloaded: /tmp/staged/1-worker-13-1662108752054159145 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 08:52:39 control response channel closed
22/09/02 08:52:40 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 08:52:40 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 08:52:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 08:52:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/02 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34385 for control ...
22/09/02 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44139 for logging ...
22/09/02 08:52:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/02 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35307 for data ...
22/09/02 08:52:41 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 08:52:41 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45329 for state ...
22/09/02 08:52:41 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 08:52:41 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 08:52:41 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 08:52:41 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 373666e4d23eec053719a0b5cf49553b3a09938d7ef7cba332341f62b20f07d6. Logs:
2022/09/02 08:52:41 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-232"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:35977"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4014"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4014"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:44139"} artifact_endpoint:{url:"localhost:34333"} control_endpoint:{url:"localhost:34385"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 08:52:41 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:46625
2022/09/02 08:52:41 Downloaded: /tmp/staged/1-worker-13-1662108752054159145 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/02 08:52:41 control response channel closed
Worker exited successfully!
22/09/02 08:52:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 08:52:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 08:52:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66_52f843aa-af87-4968-88fc-858311b67ae7: Pipeline translated successfully. Computing outputs
22/09/02 08:52:42 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 08:52:42 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 08:52:43 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 08:52:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/02 08:52:43 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35217 for control ...
22/09/02 08:52:43 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45899 for logging ...
22/09/02 08:52:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/02 08:52:43 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42225 for data ...
22/09/02 08:52:43 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 08:52:43 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34135 for state ...
22/09/02 08:52:43 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 08:52:43 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 08:52:43 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 08:52:43 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9c0ed445d5630f983c7be104dea041000c6973b2a2ca7b05b9c1e8d954911d4e. Logs:
2022/09/02 08:52:42 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-232"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:35977"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4014"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4014"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45899"} artifact_endpoint:{url:"localhost:37077"} control_endpoint:{url:"localhost:35217"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/1-go-/tmp/worker-13-1662108752054159145" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 08:52:42 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:43237
2022/09/02 08:52:42 Downloaded: /tmp/staged/1-worker-13-1662108752054159145 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/02 08:52:43 control response channel closed
Worker exited successfully!
22/09/02 08:52:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66_52f843aa-af87-4968-88fc-858311b67ae7 finished.
22/09/02 08:52:43 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-232_d11556d3-0bf9-464e-8099-82201a9bfe0a.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/bbc4f6dfa119f3960756c7f1b338e22e67b1bf88013b90ea2e8f917ea95ffdd9/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/02 08:52:43 Job state: DONE
22/09/02 08:52:43 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66_52f843aa-af87-4968-88fc-858311b67ae7
22/09/02 08:52:43 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0232-jenkins-0902085233-ae783b66_52f843aa-af87-4968-88fc-858311b67ae7
--- PASS: TestEmitParDoAfterGBK (11.60s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 115.932s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 20s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/7w3uakqvy6qyo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6543
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6543/display/redirect?page=changes>
Changes:
[noreply] Minor: Fix lint failure (#22998)
------------------------------------------
[...truncated 2.52 MB...]
22/09/02 02:52:21 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46635 for data ...
22/09/02 02:52:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 02:52:21 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 02:52:21 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 02:52:21 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 02:52:21 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 9d3658b99b9029f380509adbeb0ef10c12e214fa5b3115455e81d78cb08a5990. Logs:
2022/09/02 02:52:21 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-100"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33245"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test16876"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test16876"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35665"} artifact_endpoint:{url:"localhost:34199"} control_endpoint:{url:"localhost:38253"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 02:52:21 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:35163
2022/09/02 02:52:21 Downloaded: /tmp/staged/1-worker-13-1662087136790435260 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/02 02:52:21 control response channel closed
Worker exited successfully!
22/09/02 02:52:22 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 02:52:22 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 02:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/02 02:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 02:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43657 for control ...
22/09/02 02:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39109 for logging ...
22/09/02 02:52:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/02 02:52:23 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33693 for data ...
22/09/02 02:52:23 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 02:52:23 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 02:52:23 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 02:52:23 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 02:52:23 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f972ff9a969d01ae061bd201c5e24261adaad75274665c1da19e48d001e9a9be. Logs:
2022/09/02 02:52:22 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-100"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33245"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test16876"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test16876"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39109"} artifact_endpoint:{url:"localhost:33323"} control_endpoint:{url:"localhost:43657"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 02:52:22 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:43085
2022/09/02 02:52:23 Downloaded: /tmp/staged/1-worker-13-1662087136790435260 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/02 02:52:23 control response channel closed
Worker exited successfully!
22/09/02 02:52:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 02:52:24 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 02:52:24 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 02:52:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/02 02:52:24 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38491 for control ...
22/09/02 02:52:24 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33873 for logging ...
22/09/02 02:52:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/02 02:52:24 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37531 for data ...
22/09/02 02:52:24 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 02:52:24 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43421 for state ...
22/09/02 02:52:24 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 02:52:24 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 02:52:24 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 02:52:24 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 03a7ff39f1bdd4308db514762e6ed46e01cf950265122a15ea6a9a70a96e9ba7. Logs:
2022/09/02 02:52:24 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-100"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33245"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test16876"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test16876"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33873"} artifact_endpoint:{url:"localhost:36979"} control_endpoint:{url:"localhost:38491"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 02:52:24 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:42267
2022/09/02 02:52:24 Downloaded: /tmp/staged/1-worker-13-1662087136790435260 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/02 02:52:24 control response channel closed
22/09/02 02:52:25 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 02:52:25 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 02:52:25 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e_093e86e1-9b7a-47fa-a03e-5fb26797a763: Pipeline translated successfully. Computing outputs
22/09/02 02:52:25 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/02 02:52:26 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/02 02:52:26 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/02 02:52:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/02 02:52:26 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35587 for control ...
22/09/02 02:52:26 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34959 for logging ...
22/09/02 02:52:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/02 02:52:26 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46675 for data ...
22/09/02 02:52:26 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/02 02:52:26 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46175 for state ...
22/09/02 02:52:26 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/02 02:52:26 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/02 02:52:26 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/02 02:52:26 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 8298bb9183f23fa0a30b78b619c099198f2906c60777036481bf69eb5f852daf. Logs:
2022/09/02 02:52:26 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-100"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:33245"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test16876"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test16876"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34959"} artifact_endpoint:{url:"localhost:43877"} control_endpoint:{url:"localhost:35587"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/1-go-/tmp/worker-13-1662087136790435260" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/02 02:52:26 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:35605
2022/09/02 02:52:26 Downloaded: /tmp/staged/1-worker-13-1662087136790435260 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/02 02:52:26 control response channel closed
Worker exited successfully!
22/09/02 02:52:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e_093e86e1-9b7a-47fa-a03e-5fb26797a763 finished.
22/09/02 02:52:27 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-100_7403463e-79ca-4229-97ba-b374b5a770e4.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/d39b4af707227407214e1f5f78ff4b135eff1d6576b8216216cb91accb9aa3ca/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/02 02:52:27 Job state: DONE
22/09/02 02:52:27 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e_093e86e1-9b7a-47fa-a03e-5fb26797a763
22/09/02 02:52:27 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0100-jenkins-0902025218-90ea1d4e_093e86e1-9b7a-47fa-a03e-5fb26797a763
--- PASS: TestEmitParDoAfterGBK (10.36s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 113.899s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 1s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/riqzkg4kumaic
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6542
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6542/display/redirect?page=changes>
Changes:
[noreply] Improve BQ test utils to support JSON in a more simple manner (#22942)
[noreply] [fixes #22980] Migrate BeamFnLoggingClient to the new execution state
[Robert Bradshaw] Update proto generation script due to BEAM-13939.
[Robert Bradshaw] Regenerate typescript protos.
[noreply] Add initial read_gbq wrapper (#22616)
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 20:53:07 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 20:53:07 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 20:53:08 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c913d074fe9415b3bf587e010fa88b82b7b5b681e4e71ad8e030a4dd5ce38d62. Logs:
2022/09/01 20:53:07 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-321"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55815"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6472"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6472"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40525"} artifact_endpoint:{url:"localhost:33197"} control_endpoint:{url:"localhost:43513"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 20:53:07 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:44549
2022/09/01 20:53:07 Downloaded: /tmp/staged/1-worker-13-1662065583084993017 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 20:53:07 control response channel closed
22/09/01 20:53:08 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 20:53:09 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 20:53:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/01 20:53:09 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 20:53:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38025 for control ...
22/09/01 20:53:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39395 for logging ...
22/09/01 20:53:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/01 20:53:09 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45361 for data ...
22/09/01 20:53:09 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 20:53:09 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 20:53:09 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 20:53:09 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 20:53:09 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e653e2bed9cee48923b161917b1d2f127f22692cccc569ec92181ca03369a17b. Logs:
2022/09/01 20:53:09 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-321"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55815"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6472"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6472"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39395"} artifact_endpoint:{url:"localhost:43939"} control_endpoint:{url:"localhost:38025"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 20:53:09 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:46671
2022/09/01 20:53:09 Downloaded: /tmp/staged/1-worker-13-1662065583084993017 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 20:53:09 control response channel closed
22/09/01 20:53:10 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 20:53:10 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 20:53:11 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 20:53:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/01 20:53:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39627 for control ...
22/09/01 20:53:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35551 for logging ...
22/09/01 20:53:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/01 20:53:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37937 for data ...
22/09/01 20:53:11 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 20:53:11 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39005 for state ...
22/09/01 20:53:11 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 20:53:11 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 20:53:11 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 20:53:11 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 819253821c7bb1cd1fb9f71385bf72116cc231715c636f7712597030346e567c. Logs:
2022/09/01 20:53:10 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-321"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55815"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6472"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6472"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35551"} artifact_endpoint:{url:"localhost:33459"} control_endpoint:{url:"localhost:39627"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 20:53:10 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:38453
2022/09/01 20:53:11 Downloaded: /tmp/staged/1-worker-13-1662065583084993017 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 20:53:11 control response channel closed
22/09/01 20:53:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 20:53:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 20:53:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d_704bd18d-aa5b-4c4b-b28e-475d83ff17f0: Pipeline translated successfully. Computing outputs
22/09/01 20:53:11 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 20:53:12 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 20:53:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/01 20:53:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 20:53:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36979 for control ...
22/09/01 20:53:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37965 for logging ...
22/09/01 20:53:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/01 20:53:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43817 for data ...
22/09/01 20:53:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 20:53:12 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39561 for state ...
22/09/01 20:53:12 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 20:53:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 20:53:12 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 20:53:13 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container af5b2ca06aa11631089fecea7281b3932eca8b9091fa4bfe2768fde63b20c926. Logs:
2022/09/01 20:53:12 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-321"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:55815"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6472"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6472"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37965"} artifact_endpoint:{url:"localhost:32955"} control_endpoint:{url:"localhost:36979"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/1-go-/tmp/worker-13-1662065583084993017" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 20:53:12 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:37243
2022/09/01 20:53:12 Downloaded: /tmp/staged/1-worker-13-1662065583084993017 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/01 20:53:12 control response channel closed
Worker exited successfully!
22/09/01 20:53:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d_704bd18d-aa5b-4c4b-b28e-475d83ff17f0 finished.
22/09/01 20:53:13 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-321_c53f227c-b9d5-469d-97fc-3c9b626d863e.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/35225fd127a7285012534a162b769e465a096c388185e452195cda3090861185/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/01 20:53:13 Job state: DONE
22/09/01 20:53:13 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d_704bd18d-aa5b-4c4b-b28e-475d83ff17f0
22/09/01 20:53:13 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0321-jenkins-0901205304-d0e3fc6d_704bd18d-aa5b-4c4b-b28e-475d83ff17f0
--- PASS: TestEmitParDoAfterGBK (10.55s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 118.672s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 48s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/uf6ulsidrhtnw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6541
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6541/display/redirect?page=changes>
Changes:
[noreply] Add some explanatory comments to the wordcount registration (#22989)
[noreply] Move Go examples under the cookbook directory to generic registration
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 14:54:35 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 14:54:35 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 14:54:35 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 637a1683f2b781767cb768be5296d3c62f289f01d20c6d09103d35457b7601e0. Logs:
2022/09/01 14:54:35 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-920"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:60355"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29489"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29489"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45439"} artifact_endpoint:{url:"localhost:36555"} control_endpoint:{url:"localhost:36815"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 14:54:35 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:44741
2022/09/01 14:54:35 Downloaded: /tmp/staged/1-worker-13-1662044070944669239 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 14:54:35 control response channel closed
22/09/01 14:54:36 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 14:54:36 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 14:54:38 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 14:54:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/01 14:54:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:32963 for control ...
22/09/01 14:54:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36819 for logging ...
22/09/01 14:54:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/01 14:54:38 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34367 for data ...
22/09/01 14:54:38 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 14:54:38 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 14:54:38 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 14:54:38 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 14:54:38 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 0735dce39cadca87d0071194ef4390f45ec047b88c16672a88a9bd1b9ce83df6. Logs:
2022/09/01 14:54:38 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-920"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:60355"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29489"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29489"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36819"} artifact_endpoint:{url:"localhost:33649"} control_endpoint:{url:"localhost:32963"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 14:54:38 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:36153
2022/09/01 14:54:38 Downloaded: /tmp/staged/1-worker-13-1662044070944669239 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 14:54:38 control response channel closed
22/09/01 14:54:39 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 14:54:40 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 14:54:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/01 14:54:40 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 14:54:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38689 for control ...
22/09/01 14:54:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35947 for logging ...
22/09/01 14:54:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/01 14:54:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44971 for data ...
22/09/01 14:54:40 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 14:54:40 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38171 for state ...
22/09/01 14:54:40 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 14:54:40 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 14:54:40 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 14:54:40 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c6c81b8648d5083941b593974179d9186958665f8e0e2c8c5c4f6660dd5e8f76. Logs:
2022/09/01 14:54:40 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-920"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:60355"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29489"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29489"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35947"} artifact_endpoint:{url:"localhost:43747"} control_endpoint:{url:"localhost:38689"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 14:54:40 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:44085
2022/09/01 14:54:40 Downloaded: /tmp/staged/1-worker-13-1662044070944669239 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 14:54:40 control response channel closed
22/09/01 14:54:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 14:54:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 14:54:41 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95_c0fa456a-870c-4cea-bb45-4304580fb1bd: Pipeline translated successfully. Computing outputs
22/09/01 14:54:41 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 14:54:41 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 14:54:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 14:54:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/01 14:54:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41295 for control ...
22/09/01 14:54:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40479 for logging ...
22/09/01 14:54:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/01 14:54:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44487 for data ...
22/09/01 14:54:42 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 14:54:42 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38817 for state ...
22/09/01 14:54:42 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 14:54:42 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 14:54:42 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 14:54:42 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d259b072dce00dc960b2cad82f52f6ede436809a3bf620da81b5975f06522461. Logs:
2022/09/01 14:54:41 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-920"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:60355"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test29489"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test29489"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40479"} artifact_endpoint:{url:"localhost:39277"} control_endpoint:{url:"localhost:41295"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/1-go-/tmp/worker-13-1662044070944669239" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 14:54:41 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:45887
2022/09/01 14:54:42 Downloaded: /tmp/staged/1-worker-13-1662044070944669239 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/01 14:54:42 control response channel closed
Worker exited successfully!
22/09/01 14:54:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95_c0fa456a-870c-4cea-bb45-4304580fb1bd finished.
22/09/01 14:54:42 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-920_4faa2e21-622b-4c08-8b9c-7bb25fc76473.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/c2662acd3055aca9bd760bdae5bb29b62a797b7a9065d82167a417ff107d15dd/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/01 14:54:42 Job state: DONE
22/09/01 14:54:42 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95_c0fa456a-870c-4cea-bb45-4304580fb1bd
22/09/01 14:54:42 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0920-jenkins-0901145432-2ba48e95_c0fa456a-870c-4cea-bb45-4304580fb1bd
--- PASS: TestEmitParDoAfterGBK (11.81s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 139.771s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 15m 19s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/q4fjm6vbgixju
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6540
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6540/display/redirect>
Changes:
------------------------------------------
[...truncated 2.52 MB...]
22/09/01 08:52:13 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33211 for data ...
22/09/01 08:52:13 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 08:52:13 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 08:52:13 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 08:52:13 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 08:52:13 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6b1407d41aeed550c3308a8719572bb392a3ea77201ca553d7131f2310c5355c. Logs:
2022/09/01 08:52:12 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-915"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:56509"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4597"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4597"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41757"} artifact_endpoint:{url:"localhost:45459"} control_endpoint:{url:"localhost:39933"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 08:52:12 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:42917
2022/09/01 08:52:12 Downloaded: /tmp/staged/1-worker-13-1662022328197793611 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/01 08:52:13 control response channel closed
Worker exited successfully!
22/09/01 08:52:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 08:52:15 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 08:52:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/01 08:52:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 08:52:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36685 for control ...
22/09/01 08:52:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33863 for logging ...
22/09/01 08:52:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/01 08:52:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42755 for data ...
22/09/01 08:52:15 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 08:52:15 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 08:52:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 08:52:15 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 08:52:15 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c56d305a906eb40f1e49587d933d247f656cd9c9a51f9fd4b3608e3fe4bcd032. Logs:
2022/09/01 08:52:15 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-915"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:56509"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4597"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4597"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:33863"} artifact_endpoint:{url:"localhost:45279"} control_endpoint:{url:"localhost:36685"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 08:52:15 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:38277
2022/09/01 08:52:15 Downloaded: /tmp/staged/1-worker-13-1662022328197793611 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 08:52:15 control response channel closed
22/09/01 08:52:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 08:52:16 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 08:52:17 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 08:52:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/01 08:52:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35415 for control ...
22/09/01 08:52:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42299 for logging ...
22/09/01 08:52:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/01 08:52:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37819 for data ...
22/09/01 08:52:17 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 08:52:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35479 for state ...
22/09/01 08:52:17 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 08:52:17 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 08:52:17 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 08:52:17 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 5b0bdf348409e95e56fc4eca1005bf7047fb7c01629d2e5b5827c9d199d67569. Logs:
2022/09/01 08:52:17 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-915"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:56509"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4597"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4597"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:42299"} artifact_endpoint:{url:"localhost:33877"} control_endpoint:{url:"localhost:35415"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 08:52:17 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:37617
2022/09/01 08:52:17 Downloaded: /tmp/staged/1-worker-13-1662022328197793611 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 08:52:17 control response channel closed
22/09/01 08:52:17 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 08:52:17 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 08:52:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af_532704d0-1fed-43fc-abf8-87ac4dc3ce86: Pipeline translated successfully. Computing outputs
22/09/01 08:52:17 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 08:52:18 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 08:52:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/09/01 08:52:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 08:52:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38529 for control ...
22/09/01 08:52:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35211 for logging ...
22/09/01 08:52:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/09/01 08:52:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35291 for data ...
22/09/01 08:52:18 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 08:52:18 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45237 for state ...
22/09/01 08:52:18 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 08:52:18 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 08:52:18 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 08:52:18 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 130524d4d4f109dcb09db31925d85d9d5721181e1dcb1029abb4bc9eac583456. Logs:
2022/09/01 08:52:18 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-915"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:56509"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test4597"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test4597"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35211"} artifact_endpoint:{url:"localhost:39349"} control_endpoint:{url:"localhost:38529"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/1-go-/tmp/worker-13-1662022328197793611" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 08:52:18 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:35321
2022/09/01 08:52:18 Downloaded: /tmp/staged/1-worker-13-1662022328197793611 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 08:52:18 control response channel closed
22/09/01 08:52:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af_532704d0-1fed-43fc-abf8-87ac4dc3ce86 finished.
22/09/01 08:52:19 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-915_0f1f7838-de9c-434b-800b-1418902eff9b.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/e869f68f8a4c86d70c4689b01a44ecc99cf72d165c0b118507f179846dd0e147/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/01 08:52:19 Job state: DONE
22/09/01 08:52:19 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af_532704d0-1fed-43fc-abf8-87ac4dc3ce86
22/09/01 08:52:19 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0915-jenkins-0901085210-ad8b56af_532704d0-1fed-43fc-abf8-87ac4dc3ce86
--- PASS: TestEmitParDoAfterGBK (11.31s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 119.058s
FAIL
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 12m 55s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/lkajcs653hv6w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6539
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6539/display/redirect?page=changes>
Changes:
[Brian Hulette] Extract utilities in dataframe.schemas
[Brian Hulette] Add pandas_type_compatibility with pandas BatchConverter implementations
[Brian Hulette] Use Batched DoFns at DataFrame API boundaries
[Brian Hulette] Move dtype conversion to pandas_type_compatibility
[Brian Hulette] Always register pandas BatchConverters
[Brian Hulette] Fix interactive runner tests
[Brian Hulette] Use pandas_type_compatibility BatchConverters for dataframe.schemas
[Brian Hulette] Skip test cases broken in pandas 1.1.x
[Brian Hulette] Address review comments
[Brian Hulette] yapf, typo in test
[Brian Hulette] Add test to reproduce https://github.com/apache/beam/issues/22854
[Brian Hulette] Exercise row coder with nested optional struct
[Brian Hulette] Make RowTypeConstraint callable
[Brian Hulette] Add test to exercise RowTypeConstraint.__call__
[noreply] Run cred rotation every month (#22977)
[noreply] [BEAM-12164] Synchronize access queue in ThroughputEstimator and
------------------------------------------
[...truncated 2.52 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 02:53:48 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 02:53:48 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 02:53:48 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 3a6d1415297c5f98b198db123b18cbd8d981b7e687f8537f180ab483ee93ab11. Logs:
2022/09/01 02:53:47 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-659"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:52313"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18339"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18339"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:45753"} artifact_endpoint:{url:"localhost:44755"} control_endpoint:{url:"localhost:35769"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 02:53:47 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:38361
2022/09/01 02:53:47 Downloaded: /tmp/staged/1-worker-13-1662000823295171009 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/09/01 02:53:48 control response channel closed
Worker exited successfully!
22/09/01 02:53:48 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 02:53:49 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 02:53:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 02:53:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/09/01 02:53:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33553 for control ...
22/09/01 02:53:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36171 for logging ...
22/09/01 02:53:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/09/01 02:53:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39967 for data ...
22/09/01 02:53:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 02:53:49 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 02:53:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 02:53:49 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 02:53:49 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container d82a9b5e218335b950a4d1576a0f1a104d3e4c38f0a5511ab6a5e897db7b75c6. Logs:
2022/09/01 02:53:49 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-659"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:52313"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18339"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18339"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36171"} artifact_endpoint:{url:"localhost:34977"} control_endpoint:{url:"localhost:33553"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 02:53:49 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:35999
2022/09/01 02:53:49 Downloaded: /tmp/staged/1-worker-13-1662000823295171009 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 02:53:49 control response channel closed
22/09/01 02:53:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 02:53:50 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 02:53:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 02:53:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/09/01 02:53:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41255 for control ...
22/09/01 02:53:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43165 for logging ...
22/09/01 02:53:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/09/01 02:53:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:36069 for data ...
22/09/01 02:53:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 02:53:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42673 for state ...
22/09/01 02:53:51 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 02:53:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 02:53:51 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 02:53:51 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 1e9769cf95c33df30c028871a1377d8130da778ef5cf0990c0c2109ee22b4bc9. Logs:
2022/09/01 02:53:50 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-659"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:52313"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18339"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18339"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43165"} artifact_endpoint:{url:"localhost:40895"} control_endpoint:{url:"localhost:41255"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 02:53:50 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:44681
2022/09/01 02:53:51 Downloaded: /tmp/staged/1-worker-13-1662000823295171009 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 02:53:51 control response channel closed
22/09/01 02:53:51 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 02:53:51 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 02:53:51 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b_49debd94-2fb7-4bfd-aae9-6339425caf17: Pipeline translated successfully. Computing outputs
22/09/01 02:53:51 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/09/01 02:53:52 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/09/01 02:53:52 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/09/01 02:53:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/09/01 02:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:44253 for control ...
22/09/01 02:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41187 for logging ...
22/09/01 02:53:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/09/01 02:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46497 for data ...
22/09/01 02:53:52 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/09/01 02:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42365 for state ...
22/09/01 02:53:52 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/09/01 02:53:52 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/09/01 02:53:52 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/09/01 02:53:52 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 220f09a040eac438c994cc4c984ca81c2f3da0436fa028a562e72f2ea24d0025. Logs:
2022/09/01 02:53:52 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-659"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:52313"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18339"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18339"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:41187"} artifact_endpoint:{url:"localhost:46403"} control_endpoint:{url:"localhost:44253"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/1-go-/tmp/worker-13-1662000823295171009" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/09/01 02:53:52 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:37269
2022/09/01 02:53:52 Downloaded: /tmp/staged/1-worker-13-1662000823295171009 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/09/01 02:53:52 control response channel closed
22/09/01 02:53:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b_49debd94-2fb7-4bfd-aae9-6339425caf17 finished.
22/09/01 02:53:53 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-659_af372d2f-42df-4a20-9944-ace1cfea5b4d.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/467982e95b9339249f01619e112014ed78718485487ad563f63b87229e3342f3/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/09/01 02:53:53 Job state: DONE
22/09/01 02:53:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b_49debd94-2fb7-4bfd-aae9-6339425caf17
22/09/01 02:53:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0659-jenkins-0901025345-f7b04e8b_49debd94-2fb7-4bfd-aae9-6339425caf17
--- PASS: TestEmitParDoAfterGBK (10.18s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 114.360s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 14m 27s
114 actionable tasks: 78 executed, 31 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/fc7ju4ijewtf4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6538
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6538/display/redirect?page=changes>
Changes:
[noreply] Add ability to remove/clear map and set state (#22938)
[noreply] Fix gpu to cpu conversion with warning logs (#22795)
[noreply] Add Go stateful DoFns to CHANGES.md and fix linting violations (#22958)
[noreply] 22805: Upgrade Jackson version from 2.13.0 to 2.13.3 (#22806)
------------------------------------------
[...truncated 2.52 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 20:53:12 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 20:53:12 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 20:53:12 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 4092306b8b19f5293feb023f7ff7da2b317097a17be069acde7714966a4ec092. Logs:
2022/08/31 20:53:12 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-883"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:54917"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test19035"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test19035"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35531"} artifact_endpoint:{url:"localhost:41849"} control_endpoint:{url:"localhost:45929"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 20:53:12 Initializing Go harness: /opt/apache/beam/boot --id=268-1 --provision_endpoint=localhost:44135
2022/08/31 20:53:12 Downloaded: /tmp/staged/1-worker-13-1661979187879137444 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 20:53:12 control response channel closed
22/08/31 20:53:13 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 20:53:13 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 20:53:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 20:53:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 269-1
22/08/31 20:53:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45735 for control ...
22/08/31 20:53:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35199 for logging ...
22/08/31 20:53:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 269-2
22/08/31 20:53:14 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37375 for data ...
22/08/31 20:53:14 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 20:53:14 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 20:53:14 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 20:53:14 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 20:53:14 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f48cd500c5e6f3ab26b6cffa4f5edf95e074161decfb95fcfc0f4eb01965b8b3. Logs:
2022/08/31 20:53:13 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-883"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:54917"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test19035"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test19035"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35199"} artifact_endpoint:{url:"localhost:38129"} control_endpoint:{url:"localhost:45735"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 20:53:13 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:37579
2022/08/31 20:53:14 Downloaded: /tmp/staged/1-worker-13-1661979187879137444 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 20:53:14 control response channel closed
22/08/31 20:53:14 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 20:53:15 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 20:53:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/08/31 20:53:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 20:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41907 for control ...
22/08/31 20:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:32919 for logging ...
22/08/31 20:53:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/08/31 20:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:42675 for data ...
22/08/31 20:53:15 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 20:53:15 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43863 for state ...
22/08/31 20:53:15 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 20:53:15 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 20:53:15 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 20:53:15 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 96a53acff1de0c24baeda6ab54c039943b05421bb96ce3a93c9b736758bd49f9. Logs:
2022/08/31 20:53:15 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-883"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:54917"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test19035"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test19035"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:32919"} artifact_endpoint:{url:"localhost:43389"} control_endpoint:{url:"localhost:41907"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 20:53:15 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:39543
2022/08/31 20:53:15 Downloaded: /tmp/staged/1-worker-13-1661979187879137444 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
2022/08/31 20:53:15 control response channel closed
Worker exited successfully!
22/08/31 20:53:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 20:53:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 20:53:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0_25695592-4ad2-47b1-af28-b369b00cd573: Pipeline translated successfully. Computing outputs
22/08/31 20:53:16 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 20:53:16 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 20:53:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/08/31 20:53:17 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 20:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45717 for control ...
22/08/31 20:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37273 for logging ...
22/08/31 20:53:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/08/31 20:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43755 for data ...
22/08/31 20:53:17 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 20:53:17 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45497 for state ...
22/08/31 20:53:17 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 20:53:17 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 20:53:17 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 20:53:17 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container f26557d997b8ce7b8d8dd89dc6a1566e28ff2fa86d22e5e97252bc5343d6dbb9. Logs:
2022/08/31 20:53:17 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-883"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:54917"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test19035"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test19035"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:37273"} artifact_endpoint:{url:"localhost:36779"} control_endpoint:{url:"localhost:45717"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/1-go-/tmp/worker-13-1661979187879137444" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 20:53:17 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:33355
2022/08/31 20:53:17 Downloaded: /tmp/staged/1-worker-13-1661979187879137444 (sha256: 5a929a581a8f6a59cabd143878415a15639cdc0389a64c9fbe30fa339befbe4e, size: 26515728)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 20:53:17 control response channel closed
22/08/31 20:53:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0_25695592-4ad2-47b1-af28-b369b00cd573 finished.
22/08/31 20:53:17 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-883_ea972b1a-3110-4d42-ba19-f98f78557529.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/2c0cf584a5b91acb696eb65b729e01f099986dee50528ed6e27a8ffdd46ca5f5/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/08/31 20:53:17 Job state: DONE
22/08/31 20:53:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0_25695592-4ad2-47b1-af28-b369b00cd573
22/08/31 20:53:17 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0883-jenkins-0831205309-213865a0_25695592-4ad2-47b1-af28-b369b00cd573
--- PASS: TestEmitParDoAfterGBK (10.06s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 112.128s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 53s
114 actionable tasks: 78 executed, 31 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/xozpjc24jhzyu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6537
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6537/display/redirect?page=changes>
Changes:
[noreply] Filter out unsupported state tests (#22963)
------------------------------------------
[...truncated 2.53 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 14:58:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 14:58:49 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 14:58:49 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container fce78cdfe772eb1dee328a8f06600d270a8904cc0b9695f8339c9b9b41366598. Logs:
2022/08/31 14:58:49 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-97"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51473"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18969"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18969"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:36121"} artifact_endpoint:{url:"localhost:41127"} control_endpoint:{url:"localhost:37303"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 14:58:49 Initializing Go harness: /opt/apache/beam/boot --id=269-1 --provision_endpoint=localhost:41201
2022/08/31 14:58:49 Downloaded: /tmp/staged/1-worker-13-1661957923578884402 (sha256: efa653088ede9fc0ded9a7d7026653fd5a5940723add597335e928775ad958fb, size: 26506569)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 14:58:49 control response channel closed
22/08/31 14:58:50 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 14:58:51 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 14:58:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 270-1
22/08/31 14:58:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 14:58:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33881 for control ...
22/08/31 14:58:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39579 for logging ...
22/08/31 14:58:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 270-2
22/08/31 14:58:51 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41485 for data ...
22/08/31 14:58:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 14:58:51 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 14:58:51 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 14:58:51 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 14:58:51 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 03379ae5ee8f5b352f1777b1ec22167e3d8ebe8f7108ea7f1432f3d46e2867c3. Logs:
2022/08/31 14:58:51 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-97"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51473"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18969"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18969"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39579"} artifact_endpoint:{url:"localhost:41913"} control_endpoint:{url:"localhost:33881"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 14:58:51 Initializing Go harness: /opt/apache/beam/boot --id=270-1 --provision_endpoint=localhost:42755
2022/08/31 14:58:51 Downloaded: /tmp/staged/1-worker-13-1661957923578884402 (sha256: efa653088ede9fc0ded9a7d7026653fd5a5940723add597335e928775ad958fb, size: 26506569)
Error Setting Rlimit operation not permitted
2022/08/31 14:58:51 control response channel closed
Worker exited successfully!
22/08/31 14:58:52 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 14:58:53 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 14:58:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 271-1
22/08/31 14:58:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 14:58:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45939 for control ...
22/08/31 14:58:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38023 for logging ...
22/08/31 14:58:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 271-2
22/08/31 14:58:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37583 for data ...
22/08/31 14:58:53 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 14:58:53 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37397 for state ...
22/08/31 14:58:53 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 14:58:53 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 14:58:53 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 14:58:53 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b9d91b111e422e0fe8ead050967aea40545a5acbf04f8f6559a61ada676023d2. Logs:
2022/08/31 14:58:53 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-97"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51473"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18969"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18969"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38023"} artifact_endpoint:{url:"localhost:42523"} control_endpoint:{url:"localhost:45939"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 14:58:53 Initializing Go harness: /opt/apache/beam/boot --id=271-1 --provision_endpoint=localhost:33801
2022/08/31 14:58:53 Downloaded: /tmp/staged/1-worker-13-1661957923578884402 (sha256: efa653088ede9fc0ded9a7d7026653fd5a5940723add597335e928775ad958fb, size: 26506569)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 14:58:53 control response channel closed
22/08/31 14:58:54 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 14:58:54 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 14:58:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f_963296bc-2341-4bd3-a044-1d53e0bbc368: Pipeline translated successfully. Computing outputs
22/08/31 14:58:54 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 14:58:55 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 14:58:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 14:58:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 272-1
22/08/31 14:58:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:46661 for control ...
22/08/31 14:58:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43615 for logging ...
22/08/31 14:58:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 272-2
22/08/31 14:58:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34109 for data ...
22/08/31 14:58:55 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 14:58:55 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41541 for state ...
22/08/31 14:58:55 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 14:58:55 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 14:58:55 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 14:58:55 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 655d6494a9386d6811136920115379e88f7bd07409435fe99bb50aa967570de7. Logs:
2022/08/31 14:58:55 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-97"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:51473"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test18969"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test18969"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f"}} fields:{key:"beam:option:options_id:v1" value:{number_value:39}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:43615"} artifact_endpoint:{url:"localhost:38129"} control_endpoint:{url:"localhost:46661"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/1-go-/tmp/worker-13-1661957923578884402" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 14:58:55 Initializing Go harness: /opt/apache/beam/boot --id=272-1 --provision_endpoint=localhost:42303
2022/08/31 14:58:55 Downloaded: /tmp/staged/1-worker-13-1661957923578884402 (sha256: efa653088ede9fc0ded9a7d7026653fd5a5940723add597335e928775ad958fb, size: 26506569)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 14:58:55 control response channel closed
22/08/31 14:58:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f_963296bc-2341-4bd3-a044-1d53e0bbc368 finished.
22/08/31 14:58:56 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-97_792bc4ea-d068-45c4-ad51-60078561c309.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/d45b1229170e04bfef6138fabf8b00d78d3dbe54d26dfaaada83466d760c434f/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/08/31 14:58:56 Job state: DONE
22/08/31 14:58:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f_963296bc-2341-4bd3-a044-1d53e0bbc368
22/08/31 14:58:56 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk097-jenkins-0831145845-941ae23f_963296bc-2341-4bd3-a044-1d53e0bbc368
--- PASS: TestEmitParDoAfterGBK (12.72s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 116.002s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 13m 50s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/cialh4ceqofy6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6536
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6536/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote call on apache-beam-jenkins-10 failed
at org.jenkinsci.plugins.envinject.service.EnvironmentVariablesNodeLoader.gatherEnvVarsForNode(EnvironmentVariablesNodeLoader.java:86)
at org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:78)
at org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:42)
at hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:613)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:481)
at hudson.model.Run.execute(Run.java:1896)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Caused by: java.io.IOException: Remote call on apache-beam-jenkins-10 failed
at hudson.remoting.Channel.call(Channel.java:1004)
at hudson.FilePath.act(FilePath.java:1285)
at org.jenkinsci.plugins.envinject.service.EnvironmentVariablesNodeLoader.gatherEnvVarsForNode(EnvironmentVariablesNodeLoader.java:61)
... 8 more
Caused by: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:798)
at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:937)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1354)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
at jenkins.util.InterceptingExecutorService.submit(InterceptingExecutorService.java:36)
at jenkins.util.InterceptingExecutorService.submit(InterceptingExecutorService.java:36)
at hudson.remoting.DelegatingExecutorService.submit(DelegatingExecutorService.java:52)
at hudson.remoting.InterceptingExecutorService.submit(InterceptingExecutorService.java:50)
at hudson.remoting.InterceptingExecutorService.submit(InterceptingExecutorService.java:44)
at org.jenkinsci.remoting.util.AnonymousClassWarnings.check(AnonymousClassWarnings.java:66)
at hudson.remoting.MultiClassLoaderSerializer$Output.annotateClass(MultiClassLoaderSerializer.java:48)
at java.base/java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1281)
at java.base/java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1222)
at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1418)
at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1169)
at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:345)
at hudson.remoting.UserRequest._serialize(UserRequest.java:263)
at hudson.remoting.UserRequest.serialize(UserRequest.java:272)
at hudson.remoting.UserRequest.<init>(UserRequest.java:101)
at hudson.remoting.Channel.call(Channel.java:998)
... 10 more
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Go_VR_Spark #6535
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6535/display/redirect>
Changes:
------------------------------------------
[...truncated 2.83 MB...]
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 08:53:47 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 08:53:47 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 08:53:47 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container c4f976cde5c3e284faaf56258bbf735d5d1054254edf07ef0d20bdc3e1dd3506. Logs:
2022/08/31 08:53:47 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-409"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6117"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6117"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39131"} artifact_endpoint:{url:"localhost:45487"} control_endpoint:{url:"localhost:45715"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 08:53:47 Initializing Go harness: /opt/apache/beam/boot --id=274-1 --provision_endpoint=localhost:40403
2022/08/31 08:53:47 Downloaded: /tmp/staged/1-worker-13-1661936022764840273 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 08:53:47 control response channel closed
22/08/31 08:53:48 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 08:53:48 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 08:53:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 275-1
22/08/31 08:53:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 08:53:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39983 for control ...
22/08/31 08:53:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39337 for logging ...
22/08/31 08:53:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 275-2
22/08/31 08:53:49 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41863 for data ...
22/08/31 08:53:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 08:53:49 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 08:53:49 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 08:53:49 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 08:53:49 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 780c9d80a8c8e02648fe92211c7a71713e9eea7f27fe4608b6641a07991d059e. Logs:
2022/08/31 08:53:48 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-409"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6117"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6117"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39337"} artifact_endpoint:{url:"localhost:36503"} control_endpoint:{url:"localhost:39983"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 08:53:48 Initializing Go harness: /opt/apache/beam/boot --id=275-1 --provision_endpoint=localhost:37049
2022/08/31 08:53:49 Downloaded: /tmp/staged/1-worker-13-1661936022764840273 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 08:53:49 control response channel closed
22/08/31 08:53:49 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 08:53:50 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 08:53:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 08:53:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 276-1
22/08/31 08:53:50 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:32801 for control ...
22/08/31 08:53:50 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40737 for logging ...
22/08/31 08:53:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 276-2
22/08/31 08:53:50 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:38213 for data ...
22/08/31 08:53:50 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 08:53:50 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45073 for state ...
22/08/31 08:53:50 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 08:53:50 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 08:53:50 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 08:53:50 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 821fce6f11c9efc1e0f722921c84d9e2a8f355f9bf491cd34ade8d62bd99376f. Logs:
2022/08/31 08:53:50 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-409"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6117"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6117"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:40737"} artifact_endpoint:{url:"localhost:36883"} control_endpoint:{url:"localhost:32801"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 08:53:50 Initializing Go harness: /opt/apache/beam/boot --id=276-1 --provision_endpoint=localhost:33977
2022/08/31 08:53:50 Downloaded: /tmp/staged/1-worker-13-1661936022764840273 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 08:53:50 control response channel closed
22/08/31 08:53:51 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 08:53:51 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 08:53:51 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936_65288235-a297-4a5f-b01f-667f8d50d702: Pipeline translated successfully. Computing outputs
22/08/31 08:53:51 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
22/08/31 08:53:51 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found
22/08/31 08:53:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 277-1
22/08/31 08:53:52 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected.
22/08/31 08:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:37183 for control ...
22/08/31 08:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39047 for logging ...
22/08/31 08:53:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 277-2
22/08/31 08:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45473 for data ...
22/08/31 08:53:52 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
22/08/31 08:53:52 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33325 for state ...
22/08/31 08:53:52 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1"
payload: "\n\026apache/beam_go_sdk:dev"
capabilities: "beam:protocol:progress_reporting:v1"
capabilities: "beam:protocol:multi_core_bundle_processing:v1"
capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
capabilities: "beam:protocol:worker_status:v1"
capabilities: "beam:protocol:monitoring_info_short_ids:v1"
capabilities: "beam:version:sdk_base:go"
capabilities: "beam:coder:bytes:v1"
capabilities: "beam:coder:bool:v1"
capabilities: "beam:coder:varint:v1"
capabilities: "beam:coder:double:v1"
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies {
type_urn: "beam:artifact:type:file:v1"
type_payload: "\n\203\001/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273"
role_urn: "beam:artifact:role:go_worker_binary:v1"
}
22/08/31 08:53:52 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown.
22/08/31 08:53:52 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
22/08/31 08:53:52 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6e4b0df22d625c1bef1df1b850ddd59a61038ebad883bbb86ae0e43e59532793. Logs:
2022/08/31 08:53:52 Provision info:
pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-409"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:46093"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test6117"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test6117"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"3h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936"}} fields:{key:"beam:option:options_id:v1" value:{number_value:41}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:39047"} artifact_endpoint:{url:"localhost:34953"} control_endpoint:{url:"localhost:37183"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/1-go-/tmp/worker-13-1661936022764840273" role_urn:"beam:artifact:role:go_worker_binary:v1"}
2022/08/31 08:53:52 Initializing Go harness: /opt/apache/beam/boot --id=277-1 --provision_endpoint=localhost:38467
2022/08/31 08:53:52 Downloaded: /tmp/staged/1-worker-13-1661936022764840273 (sha256: 4a2c66409b470c44ad909bcfc1ee28b5efaa915272490d582b9c0005fa0d1833, size: 26506529)
Error Setting Rlimit operation not permitted
Worker exited successfully!
2022/08/31 08:53:52 control response channel closed
22/08/31 08:53:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936_65288235-a297-4a5f-b01f-667f8d50d702 finished.
22/08/31 08:53:53 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-409_0244d5dc-dbf9-449d-af8a-dc570f792b1e.
java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/413c27477899609efa1170f99dd60accc9aa44de200c132437a24deba10272f7/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.>
at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105)
at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180)
at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115)
at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261)
at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249)
at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115)
at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2022/08/31 08:53:53 Job state: DONE
22/08/31 08:53:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936_65288235-a297-4a5f-b01f-667f8d50d702
22/08/31 08:53:53 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0409-jenkins-0831085344-efb2f936_65288235-a297-4a5f-b01f-667f8d50d702
--- PASS: TestEmitParDoAfterGBK (10.32s)
FAIL
FAIL github.com/apache/beam/sdks/v2/go/test/regression 113.542s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task :sdks:go:test:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121
* What went wrong:
Execution failed for task ':sdks:go:test:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 14m 27s
114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/3p3roa3jvfml4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org