You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/11/24 18:39:00 UTC
Build failed in Jenkins: beam_PostCommit_XVR_Dataflow #1480
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/1480/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-13015] Simplify
------------------------------------------
[...truncated 40.95 KB...]
Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB)
Collecting distlib==0.3.1
Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting packaging>=14
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting py>=1.4.17
Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting pluggy>=0.12.0
Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting toml>=0.9.4
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting importlib-metadata<3,>=0.12
Using cached importlib_metadata-2.1.2-py2.py3-none-any.whl (10 kB)
Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Using cached virtualenv-20.10.0-py2.py3-none-any.whl (5.6 MB)
Collecting filelock>=3.0.0
Using cached filelock-3.4.0-py3-none-any.whl (9.8 kB)
Collecting six>=1.14.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting protobuf<4.0dev,>=3.5.0.post1
Using cached protobuf-3.19.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Collecting grpcio>=1.37.0
Using cached grpcio-1.42.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.0 MB)
Collecting zipp>=0.5
Using cached zipp-3.6.0-py3-none-any.whl (5.3 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.6-py3-none-any.whl (97 kB)
Collecting backports.entry-points-selectable>=1.0.4
Using cached backports.entry_points_selectable-1.1.1-py2.py3-none-any.whl (6.2 kB)
go: downloading github.com/golang/protobuf v1.5.2
go: downloading google.golang.org/grpc v1.39.0
go: downloading google.golang.org/protobuf v1.27.1
go: downloading google.golang.org/genproto v0.0.0-20210728212813-7823e685a01f
go: downloading golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c
go: downloading golang.org/x/net v0.0.0-20210423184538-5f58ad60dda6
go: downloading golang.org/x/text v0.3.6
> Task :sdks:java:container:generateLicenseReport
> Task :release:go-licenses:py:dockerRun
+ rm -rf '/output/*'
+ export GO111MODULE=off
+ GO111MODULE=off
+ go get github.com/apache/beam/sdks/python/container
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:python:setupVirtualenv
Collecting platformdirs<3,>=2
Using cached platformdirs-2.4.0-py3-none-any.whl (14 kB)
Collecting importlib-resources>=1.0
Using cached importlib_resources-5.4.0-py3-none-any.whl (28 kB)
Installing collected packages: zipp, importlib-metadata, six, pyparsing, platformdirs, importlib-resources, filelock, distlib, backports.entry-points-selectable, virtualenv, toml, setuptools, py, protobuf, pluggy, packaging, grpcio, wheel, tox, mypy-protobuf, grpcio-tools
Attempting uninstall: setuptools
Found existing installation: setuptools 40.6.2
Uninstalling setuptools-40.6.2:
Successfully uninstalled setuptools-40.6.2
Successfully installed backports.entry-points-selectable-1.1.1 distlib-0.3.1 filelock-3.4.0 grpcio-1.42.0 grpcio-tools-1.37.0 importlib-metadata-2.1.2 importlib-resources-5.4.0 mypy-protobuf-1.18 packaging-21.3 platformdirs-2.4.0 pluggy-1.0.0 protobuf-3.19.1 py-1.11.0 pyparsing-3.0.6 setuptools-59.2.0 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.10.0 wheel-0.37.0 zipp-3.6.0
> Task :release:go-licenses:java:dockerRun
+ rm -rf '/output/*'
+ export GO111MODULE=off
+ GO111MODULE=off
+ go get github.com/apache/beam/sdks/java/container
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java8:copySdkHarnessLauncher
> Task :sdks:python:sdist
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages/setuptools/dist.py>:487: UserWarning: Normalizing '2.36.0.dev' to '2.36.0.dev0'
warnings.warn(tmpl.format(**locals()))
INFO:gen_protos:Regenerating Python proto definitions (no output files).
INFO:gen_protos:Found protoc_gen_mypy at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/build/gradleenv/1922375555/bin/protoc-gen-mypy>
beam_interactive_api.proto:36:1: warning: Import google/protobuf/timestamp.proto is unused.
Writing mypy to external_transforms_pb2.pyi
Writing mypy to beam_runner_api_pb2.pyi
Writing mypy to standard_window_fns_pb2.pyi
Writing mypy to metrics_pb2.pyi
Writing mypy to endpoints_pb2.pyi
Writing mypy to schema_pb2.pyi
Writing mypy to beam_artifact_api_pb2.pyi
Writing mypy to beam_expansion_api_pb2.pyi
Writing mypy to beam_job_api_pb2.pyi
Writing mypy to beam_provision_api_pb2.pyi
Writing mypy to beam_fn_api_pb2.pyi
Writing mypy to beam_interactive_api_pb2.pyi
INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/portability/api/metrics_pb2_urns.py>
INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/portability/api/beam_artifact_api_pb2_urns.py>
INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/portability/api/standard_window_fns_pb2_urns.py>
INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/portability/api/external_transforms_pb2_urns.py>
INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/portability/api/beam_runner_api_pb2_urns.py>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
warning: no files found matching 'LICENSE.python'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md
> Task :sdks:python:container:py36:copyDockerfileDependencies
> Task :sdks:java:harness:compileJava
> Task :sdks:python:installGcpTest
> Task :sdks:java:harness:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :sdks:java:harness:classes
> Task :sdks:java:harness:jar
> Task :sdks:java:harness:shadowJar FROM-CACHE
> Task :runners:java-fn-execution:compileJava FROM-CACHE
> Task :runners:java-fn-execution:classes UP-TO-DATE
> Task :runners:java-fn-execution:jar
> Task :sdks:java:expansion-service:compileJava FROM-CACHE
> Task :sdks:java:expansion-service:classes UP-TO-DATE
> Task :sdks:java:expansion-service:jar
> Task :sdks:java:testing:expansion-service:compileTestJava FROM-CACHE
> Task :sdks:java:testing:expansion-service:testClasses
> Task :sdks:java:io:kafka:compileJava FROM-CACHE
> Task :sdks:java:io:kafka:classes UP-TO-DATE
> Task :sdks:java:io:kafka:jar
> Task :sdks:java:io:google-cloud-platform:compileJava FROM-CACHE
> Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:jar
> Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE
> Task :runners:google-cloud-dataflow-java:classes
> Task :sdks:java:container:java8:copyDockerfileDependencies
> Task :sdks:java:container:pullLicenses
Copying already-fetched licenses from <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/reports/dependency-license> to <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/target/java_third_party_licenses>
> Task :runners:google-cloud-dataflow-java:jar
> Task :runners:google-cloud-dataflow-java:worker:compileJava FROM-CACHE
> Task :runners:google-cloud-dataflow-java:worker:classes
> Task :sdks:java:testing:expansion-service:buildTestExpansionServiceJar FROM-CACHE
> Task :sdks:java:container:pullLicenses
Collecting beautifulsoup4<5.0,>=4.9.0 (from -r <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/license_scripts/requirement.txt> (line 19))
Using cached https://files.pythonhosted.org/packages/69/bf/f0f194d3379d3f3347478bd267f754fc68c11cbf2fe302a6ab69447b1417/beautifulsoup4-4.10.0-py3-none-any.whl
Collecting pyyaml<6.0.0,>=3.12 (from -r <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/license_scripts/requirement.txt> (line 20))
Using cached https://files.pythonhosted.org/packages/7a/5b/bc0b5ab38247bba158504a410112b6c03f153c652734ece1849749e5f518/PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting tenacity<6.0,>=5.0.2 (from -r <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/license_scripts/requirement.txt> (line 21))
Using cached https://files.pythonhosted.org/packages/45/67/67bb1db087678bc5c6f20766cf18914dfe37b0b9d4e4c5bb87408460b75f/tenacity-5.1.5-py2.py3-none-any.whl
Collecting soupsieve>1.2 (from beautifulsoup4<5.0,>=4.9.0->-r <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/license_scripts/requirement.txt> (line 19))
Using cached https://files.pythonhosted.org/packages/72/a6/fd01694427f1c3fcadfdc5f1de901b813b9ac756f0806ef470cfed1de281/soupsieve-2.3.1-py3-none-any.whl
Collecting six>=1.9.0 (from tenacity<6.0,>=5.0.2->-r <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/license_scripts/requirement.txt> (line 21))
Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl
Installing collected packages: soupsieve, beautifulsoup4, pyyaml, six, tenacity
> Task :runners:google-cloud-dataflow-java:worker:shadowJar FROM-CACHE
> Task :sdks:java:container:pullLicenses
Successfully installed beautifulsoup4-4.10.0 pyyaml-5.4.1 six-1.16.0 soupsieve-2.3.1 tenacity-5.1.5
You are using pip version 18.1, however version 21.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Executing <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/reports/dependency-license/index.json> --output_dir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/target/java_third_party_licenses> --dep_url_yaml=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml>
INFO:root:Pulling license for 209 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 9.278787 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:java:container:java8:copyJavaThirdPartyLicenses
> Task :sdks:python:installGcpTest
Successfully installed apache-beam-2.36.0.dev0 atomicwrites-1.4.0 attrs-21.2.0 azure-core-1.20.1 azure-storage-blob-12.9.0 boto3-1.20.12 botocore-1.23.12 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.7 crcmod-1.7 cryptography-36.0.0 dataclasses-0.8 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.7 fasteners-0.16.3 freezegun-1.1.0 google-api-core-1.31.4 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.30.1 google-cloud-bigquery-storage-2.10.1 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.3.1 google-cloud-language-1.3.0 google-cloud-pubsub-1.7.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.53.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.0 jmespath-0.10.0 libcst-0.3.23 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.19.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.1 pandas-1.1.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.8 psycopg2-binary-2.9.2 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.1 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.3.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.26.0 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.27 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-3.10.0.2 typing-inspect-0.7.1 urllib3-1.26.7 wcwidth-0.2.5 websocket-client-1.2.1 wrapt-1.13.3
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
> Task :release:go-licenses:py:dockerRun
+ go-licenses save github.com/apache/beam/sdks/python/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/python/container
+ tee /output/licenses/list.csv
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java8:copyGolangLicenses
> Task :release:go-licenses:py:createLicenses
> Task :sdks:python:container:py36:copyGolangLicenses
> Task :sdks:java:container:java8:dockerPrepare
> Task :sdks:python:container:py36:dockerPrepare
> Task :sdks:java:container:java8:docker
> Task :sdks:python:container:py36:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
> Task :runners:google-cloud-dataflow-java:setupXVR
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerSetup
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211124180148]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24f74c3c1944ffb1ed8c188b5d30dcaa25083f2645709cabb05d7f8846d372d0]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211124180148] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24f74c3c1944ffb1ed8c188b5d30dcaa25083f2645709cabb05d7f8846d372d0])].
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24f74c3c1944ffb1ed8c188b5d30dcaa25083f2645709cabb05d7f8846d372d0
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:24f74c3c1944ffb1ed8c188b5d30dcaa25083f2645709cabb05d7f8846d372d0].
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:6fa92d250366553ec4080ac7cddd2997d1bb3cbc54b5d5b58c44792634d5912a
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Wed, 24 Nov 2021 18:38:55 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:6fa92d250366553ec4080ac7cddd2997d1bb3cbc54b5d5b58c44792634d5912a': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 282
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 37m 53s
132 actionable tasks: 95 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xvhbwsq7dtmee
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_XVR_Dataflow #1483
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/1483/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Dataflow #1482
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/1482/display/redirect>
Changes:
------------------------------------------
[...truncated 57.81 KB...]
Collecting pydot<2,>=1.2.0
Using cached pydot-1.4.2-py2.py3-none-any.whl (21 kB)
Collecting python-dateutil<3,>=2.8.0
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz>=2018.3
Using cached pytz-2021.3-py2.py3-none-any.whl (503 kB)
Collecting requests<3.0.0,>=2.24.0
Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting typing-extensions<4,>=3.7.0
Using cached typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Collecting dataclasses
Using cached dataclasses-0.8-py3-none-any.whl (19 kB)
Collecting cachetools<5,>=3.1.0
Using cached cachetools-4.2.4-py3-none-any.whl (10 kB)
Collecting google-apitools<0.5.32,>=0.5.31
Using cached google_apitools-0.5.31-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-2.3.3-py2.py3-none-any.whl (155 kB)
Collecting google-cloud-datastore<2,>=1.8.0
Using cached google_cloud_datastore-1.15.3-py2.py3-none-any.whl (134 kB)
Collecting google-cloud-pubsub<2,>=0.39.0
Using cached google_cloud_pubsub-1.7.0-py2.py3-none-any.whl (144 kB)
Collecting google-cloud-bigquery<3,>=1.6.0
Using cached google_cloud_bigquery-2.30.1-py2.py3-none-any.whl (203 kB)
Collecting google-cloud-bigquery-storage>=2.6.3
Using cached google_cloud_bigquery_storage-2.10.1-py2.py3-none-any.whl (171 kB)
Collecting google-cloud-core<2,>=0.28.1
Using cached google_cloud_core-1.7.2-py2.py3-none-any.whl (28 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=0.12.0
Using cached google_cloud_dlp-3.3.1-py2.py3-none-any.whl (110 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting pandas<1.4,>=1.0
Using cached pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl (9.5 MB)
Collecting freezegun>=0.3.12
Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (603 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.27-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
> Task :sdks:java:container:pullLicenses
INFO:root:pull_licenses_java.py succeed. It took 20.672381 seconds with 16 threads.
Copying licenses from <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:java:container:java8:copyJavaThirdPartyLicenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
> Task :release:go-licenses:py:dockerRun
+ go-licenses save github.com/apache/beam/sdks/python/container --save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/python/container
+ tee /output/licenses/list.csv
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java8:copyGolangLicenses
> Task :release:go-licenses:py:createLicenses
> Task :sdks:python:container:py36:copyGolangLicenses
> Task :sdks:python:container:py36:dockerPrepare
> Task :sdks:java:container:java8:dockerPrepare
> Task :sdks:python:container:py36:docker
> Task :sdks:java:container:java8:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
> Task :runners:google-cloud-dataflow-java:setupXVR
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerSetup
Launching Java expansion service @ 42261
Launching Python expansion service @ 39231
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --sdk_harness_container_image_overrides=.*java.*,us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125060105 --experiments=use_runner_v2 --experiments=shuffle_mode=appliance --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz>
>>> pytest options: --capture=no --numprocesses=8 --timeout=4500 --log-cli-level=INFO
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.13, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[gw0] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
[gw1] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
[gw2] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
[gw4] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
[gw3] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
[gw5] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
[gw7] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
[gw6] Python 3.6.13 (default, Feb 20 2021, 21:42:50) -- [GCC 5.4.0 20160609]
gw0 [10] / gw1 [10] / gw2 [10] / gw3 [10] / gw4 [10] / gw5 [10] / gw6 [10] / gw7 [10]
scheduling tests via LoadScheduling
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_per_key
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_group_by_key
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_cogroup_by_key
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_multi_input_output_with_sideinput
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_globally
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_class_lookup_payload_builder
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_external_transform
[gw7] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_group_by_key
[gw6] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_multi_input_output_with_sideinput
[gw1] PASSED apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_class_lookup_payload_builder
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_prefix
[gw2] PASSED apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_external_transform
[gw5] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_per_key
[gw0] PASSED apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_partition
[gw4] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_cogroup_by_key
[gw3] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_globally
[gw1] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_prefix
[gw0] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_partition
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/pytest_xlangValidateRunner.xml> -
================== 10 passed, 10 warnings in 1295.18 seconds ===================
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 4773.
Stopping expansion service pid: 4774.
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125060105
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a9da079797aa8ce6fcf59d2fc32f5d6c4fd69e6c1f5a2335116b43649ec9f715
Deleted: sha256:542b2a6a6765b09808f299dd18866da8c2ef97b1915370fec0db5197df740852
Deleted: sha256:a0f0b90b34af0ce326f9f653a78db0688506904f336e8ba79b9f1ce29c475e7b
Deleted: sha256:dff65b26948637b97e94aeb4d302ef2e028c68499ce2a7d6b82c6fce5d7037ee
Deleted: sha256:20211ed7d637558e5e2333bb66f75407b560ecb93cbbf3006edcbd9b95e49e8d
Deleted: sha256:8ce0a85c62202d0b3b7cae488e1599ef3c5599bc3236c84dd38248a2288b3429
Deleted: sha256:8617bfb4546a6ce3089ee4e5eba10daa1943057c12546ae20fb28e3974b6af04
Deleted: sha256:9a9d039e0b3e1d41b6680de776a82bb33208925ce0cd3b6d38035772dae1d2da
Deleted: sha256:8e102aa630c2dfe4947263282b7bafb232367ce6f6dc391ab4b9130d14209c7a
Deleted: sha256:ed4316952e622ba5234d0834e637467f207e0f95c1b14e473b53eb35ac4e5e84
Deleted: sha256:1e9aeaa3a1a54134ad088261dc65630c6dd2a3f1d627ec7abb82260578be568c
Deleted: sha256:cb5fd135105a71b245ad88d61b43a70822fd8ef64a723ae9b378abcd23e3853d
Deleted: sha256:5f14b2cfd3d3cb6db140c398b367c9033df8bb988e5f178f03608f8c88ba4809
Deleted: sha256:d2b86b42fa236311fe49006af25397f329f4a50e6e0186cb66c48bf38be8bff4
Deleted: sha256:6b60e59f67ec020c017f3fbbc4195c8ea81d350060d786432e82aa6a9c9b8459
Deleted: sha256:06a9c9730ea5d3e193c482bace0d57c804e2e83044a8a80fd8837d54a9ca09af
Deleted: sha256:cfa3b6640f600a402c621f3192fb800dd6dc79442b03949564fbe6d9af2edd52
Deleted: sha256:bb5779ddb51fe9e9d14625497aab91c55cabf7d0b1fc09b309ff3ecffdab51b1
Deleted: sha256:3fb0a5bbf31aac9acb65c2fb733f7b12aeedc605bbb301cb0241f7c36d9e90e7
Deleted: sha256:a4f6f3ef3fbe5c89602252d28a4f216b5cf14e1ec3fb3162230bfc35d96cb1bb
Deleted: sha256:145a3a0b2e7903d448c45115fa174777898518a12be38617b584585c336c43c1
Deleted: sha256:9fb746a72570d6907307bfaddb54a30f844030d7140c8ed3c3d4c3dab0ce680b
Deleted: sha256:3e636fd44fb34d643e40cfcc8461565158ad8228eecf56ec60ec774e69731a14
Deleted: sha256:a4ddac88f1d1527c593ff7180b5a7e5a71b1b45df58bb9b5d245158632ea6c03
Deleted: sha256:6b1710f705352bc69d13211cff9d5688d0bc0e0f31c76a0ae2ac305c4b0418a4
Deleted: sha256:61831dfb40bad93606d80c3fdc0f5cc85b707905bfef702fd096838fc9441473
Deleted: sha256:1737b0e45db46e9e020a998b8a2a77e54354b9dbf0c5ccaef357f6a74b677a20
Deleted: sha256:b4abd0e62a749e8f7bd6e0bc05e171d4f49e85fedafe562c9ec7712289390c06
Deleted: sha256:3698ec2f9580376593e3e2b404822c87ddc74ccb17c5bdc8b42c344f72893a7e
ERROR: (gcloud.container.images.untag) Image could not be found: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125060105]
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 279
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command 'gcloud'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 38m 43s
132 actionable tasks: 94 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/unl6imyy4lr56
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Dataflow #1481
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/1481/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-13015] Simplify DataStreamsDecoder by moving hasNext logic to next
------------------------------------------
[...truncated 713.07 KB...]
...021-11-25T00:26:04.990168Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f693c407c88>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed. Causes: Job appears to be stuck. Several workers have failed to start up in a row, and no worker has successfully started up for this job. Last error reported: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104 not found: manifest unknown: Failed to fetch "20211125000104" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211125000104".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image..
apache_beam/runners/dataflow/dataflow_runner.py:1636: DataflowRuntimeException
------------------------------ Captured log call -------------------------------
INFO apache_beam.runners.portability.stager:stager.py:302 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO root:environments.py:296 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python36-fnapi:beam-master-20211116
INFO root:environments.py:304 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python36-fnapi:beam-master-20211116" for Docker environment
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7f693e4aa400> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7f693e4aab70> ====================
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:454 Defaulting to the temp_location as staging_location: gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/beam-sdks-java-testing-expansion-service-testExpansionService-2.36.0-SNAPSHOT-bq7hqnIyzdvp6zqfI_Bsbz6lozRHMqcJYKEVTRrIC-M.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/beam-sdks-java-testing-expansion-service-testExpansionService-2.36.0-SNAPSHOT-bq7hqnIyzdvp6zqfI_Bsbz6lozRHMqcJYKEVTRrIC-M.jar in 4 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/dataflow_python_sdk.tar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/dataflow_python_sdk.tar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/pipeline.pb...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:667 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1125002555-864772.1637799955.865239/pipeline.pb in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:828 Create job: <Job
clientRequestId: '20211125002555866107-3474'
createTime: '2021-11-25T00:26:04.990168Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-24_16_26_01-12720373182222155331'
location: 'us-central1'
name: 'beamapp-jenkins-1125002555-864772'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-25T00:26:04.990168Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:830 Created job with id: [2021-11-24_16_26_01-12720373182222155331]
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:831 Submitted job: 2021-11-24_16_26_01-12720373182222155331
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:837 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-24_16_26_01-12720373182222155331?project=apache-beam-testing
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-11-24_16_26_01-12720373182222155331 is in state JOB_STATE_RUNNING
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:07.566Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2021-11-24_16_26_01-12720373182222155331. The number of workers will be between 1 and 1000.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:07.691Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2021-11-24_16_26_01-12720373182222155331.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:09.601Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.131Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.163Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.229Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.264Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert_that/Group/CoGroupByKeyImpl/GroupByKey: GroupByKey not followed by a combiner.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.295Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey: GroupByKey not followed by a combiner.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.331Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.362Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.392Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.431Z: JOB_MESSAGE_DEBUG: Inserted coder converter before flatten ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Flatten_27
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.461Z: JOB_MESSAGE_DEBUG: Inserted coder converter before flatten ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Flatten_27
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.498Z: JOB_MESSAGE_DETAILED: Unzipping flatten ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Flatten_27 for input ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-0-_25.None-post14
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.530Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write, through flatten assert_that/Group/CoGroupByKeyImpl/Flatten, into producer assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.564Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/MapTuple(collect_values) into assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.596Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/RestoreTags into assert_that/Group/CoGroupByKeyImpl/MapTuple(collect_values)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.628Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Unkey into assert_that/Group/RestoreTags
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.663Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Match into assert_that/Unkey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.685Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write into assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.717Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/FlatMap(<lambda at core.py:3224>) into Create/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.748Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/AddRandomKeys into Create/FlatMap(<lambda at core.py:3224>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.786Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps) into Create/MaybeReshuffle/Reshuffle/AddRandomKeys
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.837Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.875Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.901Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.932Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps) into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.963Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:10.997Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/Map(decode) into Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.023Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:prefix)/Map/ParMultiDo(Anonymous) into Create/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.077Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/WindowInto(WindowIntoFn) into ExternalTransform(beam:transforms:xlang:test:prefix)/Map/ParMultiDo(Anonymous)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.117Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Create/FlatMap(<lambda at core.py:3224>) into assert_that/Create/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.146Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Create/Map(decode) into assert_that/Create/FlatMap(<lambda at core.py:3224>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.176Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Tag[0] into assert_that/Create/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.206Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity into assert_that/Group/CoGroupByKeyImpl/Tag[0]
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.242Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/ToVoidKey into assert_that/WindowInto(WindowIntoFn)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.274Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Tag[1] into assert_that/ToVoidKey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.301Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity into assert_that/Group/CoGroupByKeyImpl/Tag[1]
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.345Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.374Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.408Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.442Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.589Z: JOB_MESSAGE_DEBUG: Executing wait step start26
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.655Z: JOB_MESSAGE_BASIC: Executing operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.687Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/CoGroupByKeyImpl/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.700Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.726Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.765Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.778Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Group/CoGroupByKeyImpl/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.821Z: JOB_MESSAGE_DEBUG: Value "assert_that/Group/CoGroupByKeyImpl/GroupByKey/Session" materialized.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.850Z: JOB_MESSAGE_DEBUG: Value "Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Session" materialized.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.877Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at core.py:3224>)+assert_that/Create/Map(decode)+assert_that/Group/CoGroupByKeyImpl/Tag[0]+assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:11.909Z: JOB_MESSAGE_BASIC: Executing operation Create/Impulse+Create/FlatMap(<lambda at core.py:3224>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:26:26.042Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:27:09.962Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:27:51.037Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104 not found: manifest unknown: Failed to fetch "20211125000104" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211125000104".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:28:19.157Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:29:02.863Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104 not found: manifest unknown: Failed to fetch "20211125000104" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211125000104".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:29:27.094Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:30:13.423Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104 not found: manifest unknown: Failed to fetch "20211125000104" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211125000104".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:30:40.125Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:31:26.287Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104 not found: manifest unknown: Failed to fetch "20211125000104" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211125000104".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:31:54.739Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:39.884Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104 not found: manifest unknown: Failed to fetch "20211125000104" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211125000104".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:39.918Z: JOB_MESSAGE_ERROR: Workflow failed. Causes: Job appears to be stuck. Several workers have failed to start up in a row, and no worker has successfully started up for this job. Last error reported: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104 not found: manifest unknown: Failed to fetch "20211125000104" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211125000104".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image..
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:39.976Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at core.py:3224>)+assert_that/Create/Map(decode)+assert_that/Group/CoGroupByKeyImpl/Tag[0]+assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:39.977Z: JOB_MESSAGE_BASIC: Finished operation Create/Impulse+Create/FlatMap(<lambda at core.py:3224>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:40.048Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:40.110Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:40.144Z: JOB_MESSAGE_BASIC: Stopping worker pool...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:32:40.485Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized worker pool from 1 to 0.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:33:37.510Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-11-25T00:33:37.538Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-11-24_16_26_01-12720373182222155331 is in state JOB_STATE_FAILED
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/pytest_xlangValidateRunner.xml> -
============== 9 failed, 1 passed, 2 warnings in 1154.85 seconds ===============
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava FAILED
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 16955.
Stopping expansion service pid: 16956.
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:170c10fa2852f947f119a61452897558b3a30d7fc5539259acb9e0d89d4201a5
Deleted: sha256:23b52149b9600646caf1af6ab71738524142269a1e4099eec650adeb433238c7
Deleted: sha256:86f05866537ba7866bbecb8e41011526df94287ae34187382e9b1b17e3af0df2
Deleted: sha256:dcc47623f039ea13bdd14acc1dd1df65497a2f9cfb53ae99375f347159989f19
Deleted: sha256:1ece5223fd60fb17feed63ecae76a6df744a8c906abe4f6f7f6b9a0e0b3a5808
Deleted: sha256:c02c48dc34aa43118b730a4980fdb118b9038a4d37dd741c97e3f36d1542757f
Deleted: sha256:31c3692e6a5a0a64e2b8a36fcbf5e9ca2f3c1f73ed6eb4b8410214497c2144fa
Deleted: sha256:11d59b966b318283b890e88b559ddecf2b70150c9758b95bacdba0c0568794e4
Deleted: sha256:62b64ae0e82696c79c5d5eaa89bef4e8710eea8c4daef3c1bbbef2ca37858a87
Deleted: sha256:3c2f4ad18a684f062c55e7ce06f046254238e5d336e8c3655d34e2d84a857cdd
Deleted: sha256:1e827c695aea6b04fedfb12f43e220e842725666f286c011f0776110db370bac
Deleted: sha256:5a2a3fb438276ec686154295ae886a547323cd34f96f6eca968022da19ebd7ee
Deleted: sha256:e2aa5f82b970e6aef66614502dab758e68c178ef20fc4ba7009d26e91e130e74
Deleted: sha256:a26e5df84ff3fa3efb23ec157e50aea2c1503a15de7dd7ed8f9172be1fc19a6a
Deleted: sha256:86c0d4307df8c1a44703a0b438afb9e7a1b3e6c9eaa40f99be92bd23ac26742c
Deleted: sha256:e10461a699abd4553d2bc88da3672fc325334265f25233cb6ae87dff1ddc01b8
Deleted: sha256:c3c51aac3eac3b24dda41753d5efbab95ed43e1d850ea332203cd1598dd9cd97
Deleted: sha256:74386f936c6c829aec06e98ff1e8ea59ecfe12be256558623939f795b44a2c97
Deleted: sha256:3af411e3e033e42cbb7228e1e327a1e6395e39d17145437b0f277ec99e46a2c4
Deleted: sha256:df74ef1649ecad6b7bb48855cacd6edf22992b58760ce198ee0d8852235523ca
Deleted: sha256:90c7903cf194160b0e06ab7c19163dd37c65fe3e07561cb0898d6ad6f253aabb
Deleted: sha256:b23e8e2d977dd23abfa5ea8f041500b27925ef36ab1a968edf531aca0b5699d5
Deleted: sha256:596e7a62fae362b79395ef36824483e311c048fd0311d24ffdeabfb5d08c3022
Deleted: sha256:955bbc7d682048b7eeb82a34ea42d45b7c81b8dffefeea670e7e8407f3c1b125
Deleted: sha256:eb6293359b23a7d0a17f1d36f233b50e7c594c55ced0734929b2db887eabb827
Deleted: sha256:fbe020e1fde73c2029a7e594dd025da83aeddb2edbb9f19256112de354bf3b3b
Deleted: sha256:c5ba7b08e68c7cafcc9803b7e835a8bfcab88cd5d0127f47e913986ab2cbdf34
Deleted: sha256:cd8edc179975a46f9358ca7c007136643c12d66e09e32513d457050721fd932f
Deleted: sha256:fe21954bf75f69edb6008bf436793d3e5a847f7473e23b8aedf63db8aa7ee56a
ERROR: (gcloud.container.images.untag) Image could not be found: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211125000104]
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 279
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command 'gcloud'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 33m 56s
132 actionable tasks: 98 executed, 30 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iqnc5oefhztde
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org