You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/04/20 12:16:02 UTC
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #128
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/128/display/redirect>
Changes:
------------------------------------------
[...truncated 95.29 KB...]
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<2,>=0.12.0
Using cached google_cloud_dlp-1.0.0-py2.py3-none-any.whl (169 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.8.0-py2.py3-none-any.whl (341 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.13.0-py2.py3-none-any.whl (133 kB)
Collecting freezegun>=0.3.12
Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting nose>=1.3.7
Using cached nose-1.3.7-py3-none-any.whl (154 kB)
Collecting nose_xunitmp>=0.4.1
Using cached nose_xunitmp-0.4.1-py3-none-any.whl
Collecting pandas<1.3.0,>=1.0
Using cached pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl (9.5 MB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<6.0.0,>=3.12
Using cached PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl (640 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.8.0-py2.py3-none-any.whl (23 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.9-cp36-cp36m-manylinux2014_x86_64.whl (1.5 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.8.6-cp36-cp36m-manylinux1_x86_64.whl (3.0 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.0-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from azure-core>=1.7.0->apache-beam==2.30.0.dev0) (1.15.0)
Collecting cryptography>=2.1.4
Using cached cryptography-3.4.7-cp36-abi3-manylinux2014_x86_64.whl (3.2 MB)
Collecting msrest>=0.6.18
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.5.0,>=0.4.0
Using cached s3transfer-0.4.0-py2.py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.21.0,>=1.20.54
Using cached botocore-1.20.54-py2.py3-none-any.whl (7.4 MB)
Collecting urllib3<1.27,>=1.25.4
Using cached urllib3-1.26.4-py2.py3-none-any.whl (153 kB)
Collecting cffi>=1.12
Using cached cffi-1.14.5-cp36-cp36m-manylinux1_x86_64.whl (401 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.16-py2.py3-none-any.whl (28 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from google-auth<2,>=1.18.0->apache-beam==2.30.0.dev0) (56.0.0)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting proto-plus>=1.10.0
Using cached proto_plus-1.18.1-py3-none-any.whl (42 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.23.0
Using cached google_api_core-1.26.3-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.30.0.dev0) (20.9)
Collecting google-resumable-media<2.0dev,>=0.6.0
Using cached google_resumable_media-1.2.0-py2.py3-none-any.whl (75 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.53.0-py2.py3-none-any.whl (198 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.1.2-cp36-cp36m-manylinux2014_x86_64.whl (38 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.30.0.dev0) (2.4.7)
Collecting pbr>=0.11
Using cached pbr-5.5.1-py2.py3-none-any.whl (106 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.0-py2.py3-none-any.whl (45 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.30.0.dev0) (0.13.1)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.30.0.dev0) (2.1.1)
Collecting attrs>=17.4.0
Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.30.0.dev0) (1.10.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.7.0-py3-none-any.whl (48 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.30.0.dev0) (3.4.1)
Collecting pytest-forked
Using cached pytest_forked-1.3.0-py2.py3-none-any.whl (4.7 kB)
Collecting execnet>=1.1
Using cached execnet-1.8.0-py2.py3-none-any.whl (39 kB)
Collecting apipkg>=1.4
Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)
Collecting chardet<5,>=3.0.2
Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting idna<3,>=2.5
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.0.0-cp36-cp36m-manylinux2010_x86_64.whl (156 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.0-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.12.1-cp36-cp36m-linux_x86_64.whl
Collecting websocket-client>=0.32.0
Using cached websocket_client-0.58.0-py2.py3-none-any.whl (61 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.30.0.dev0-py3-none-any.whl size=2448468 sha256=b1f1e685d55c5e445d1eb07fba8b9a70890dde36cf860de327fdb9fc97a6f84c
Stored in directory: /home/jenkins/.cache/pip/wheels/d8/1b/4b/f6a933018d31557ecdfd2019c766338eda6339bb0f184dcf50
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pycparser, pyasn1-modules, idna, chardet, certifi, cachetools, wcwidth, requests, pytz, python-dateutil, oauthlib, more-itertools, jmespath, googleapis-common-protos, google-auth, cffi, attrs, atomicwrites, websocket-client, requests-oauthlib, pytest, numpy, isodate, httplib2, grpcio-gcp, google-crc32c, google-api-core, docopt, botocore, apipkg, wrapt, typing-extensions, s3transfer, pytest-forked, pymongo, pydot, pyarrow, proto-plus, pbr, oauth2client, nose, msrest, hdfs, grpc-google-iam-v1, greenlet, google-resumable-media, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, dataclasses, cryptography, crcmod, azure-core, avro-python3, testcontainers, tenacity, sqlalchemy, requests-mock, pyyaml, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, nose-xunitmp, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
> Task :release:go-licenses:py:dockerRun
+ go-licenses save github.com/apache/beam/sdks/python/container --save_path=/output/licenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses
> Task :sdks:python:installGcpTest
Successfully installed apache-beam-2.30.0.dev0 apipkg-1.5 atomicwrites-1.4.0 attrs-20.3.0 avro-python3-1.9.2.1 azure-core-1.13.0 azure-storage-blob-12.8.0 boto3-1.17.54 botocore-1.20.54 cachetools-4.2.1 certifi-2020.12.5 cffi-1.14.5 chardet-4.0.0 crcmod-1.7 cryptography-3.4.7 dataclasses-0.8 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.0 docopt-0.6.2 execnet-1.8.0 fastavro-1.4.0 fasteners-0.16 freezegun-1.1.0 google-api-core-1.26.3 google-apitools-0.5.31 google-auth-1.29.0 google-cloud-bigquery-2.13.1 google-cloud-bigtable-1.7.0 google-cloud-core-1.6.0 google-cloud-datastore-1.15.3 google-cloud-dlp-1.0.0 google-cloud-language-1.3.0 google-cloud-pubsub-1.7.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.1.2 google-resumable-media-1.2.0 googleapis-common-protos-1.53.0 greenlet-1.0.0 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 hdfs-2.6.0 httplib2-0.19.1 idna-2.10 isodate-0.6.0 jmespath-0.10.0 mock-2.0.0 more-itertools-8.7.0 msrest-0.6.21 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.19.5 oauth2client-4.1.3 oauthlib-3.1.0 pandas-1.1.5 parameterized-0.7.5 pbr-5.5.1 proto-plus-1.18.1 psycopg2-binary-2.8.6 pyarrow-3.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.11.3 pytest-4.6.11 pytest-forked-1.3.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.1 pytz-2021.1 pyyaml-5.4.1 requests-2.25.1 requests-mock-1.8.0 requests-oauthlib-1.3.0 rsa-4.7.2 s3transfer-0.4.0 sqlalchemy-1.4.9 tenacity-5.1.5 testcontainers-3.4.0 typing-extensions-3.7.4.3 urllib3-1.26.4 wcwidth-0.2.5 websocket-client-0.58.0 wrapt-1.12.1
> Task :release:go-licenses:py:dockerRun
+ go-licenses csv github.com/apache/beam/sdks/python/container
+ tee /output/licenses/list.csv
github.com/apache/beam/sdks/python/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/nightlyone/lockfile,https://github.com/nightlyone/lockfile/blob/master/LICENSE,MIT
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses
> Task :sdks:go:resolveBuildDependencies
Resolving golang.org/x/text: commit='23ae387dee1f90d29a23c0e87ee0b46038fbed0e', urls=[https://go.googlesource.com/text]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
> Task :release:go-licenses:java:dockerRun
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java8:copyGolangLicenses
> Task :release:go-licenses:py:createLicenses
> Task :sdks:python:container:py36:copyGolangLicenses
> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/api: commit='0324d5e90dc7753607860272666845fad9ceb97e', urls=[https://code.googlesource.com/google-api-go-client]
Resolving google.golang.org/genproto: commit='4d944d34d83c502a5f761500a14d8842648415c3', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='5e8f83304c0563d1ba74db05fee83d9c18ab9a58', urls=[https://github.com/grpc/grpc-go]
Resolving google.golang.org/protobuf: commit='d165be301fb1e13390ad453281ded24385fd8ebc', urls=[https://go.googlesource.com/protobuf]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild
> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/go>
> Task :sdks:python:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/go>
> Task :sdks:java:container:installDependencies
> Task :sdks:python:container:installDependencies
> Task :sdks:python:container:buildDarwinAmd64
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:java8:copySdkHarnessLauncher
> Task :sdks:java:container:java8:dockerPrepare
> Task :sdks:python:container:buildLinuxAmd64
> Task :sdks:python:container:goBuild
> Task :sdks:python:container:py36:copyLauncherDependencies
> Task :sdks:python:container:py36:dockerPrepare
> Task :sdks:java:container:java8:docker
> Task :sdks:python:container:py36:docker
The command '/bin/sh -c if [ "$pull_licenses" = "true" ] ; then pip install 'pip-licenses<3.0.0;python_version>="3.5"' && pip install 'pip-licenses==1.18.0;python_version<="2.7"' && python /tmp/license_scripts/pull_licenses_py.py && pip uninstall -y pip-licenses && rm -rf /root/.cache/pip ; else rm -rf /opt/apache/beam/third_party_licenses ; fi' returned a non-zero code: 1
> Task :sdks:python:container:py36:docker FAILED
> Task :runners:spark:3:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:python:container:py36:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 15m 28s
172 actionable tasks: 121 executed, 47 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/czgn32a5rurco
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_XVR_Spark3 #136
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/136/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #135
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/135/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12191] Add a test for python template generation with upload_graph
[noreply] [BEAM-7372] fix wrong usage of with_traceback (#14566)
[noreply] [BEAM-7372] cleanup codes for py2 from apache_beam/transforms (#14544)
[noreply] [BEAM-2085] Fixups for Python resource hints. (#14605)
------------------------------------------
[...truncated 995.29 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 176, in test_windowing_before_sql
assert_that(out, equal_to([(1, ), (1, ), (1, )]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0422061834-bb3fe907_868d0915-a7ca-43af-a596-8ce31f74d501 failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '49783']
root: DEBUG: Waiting for grpc channel to be ready at localhost:49783.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:49783'
root: DEBUG: Waiting for grpc channel to be ready at localhost:49783.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:26 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:49783.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49783.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49783.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49783.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:27 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:28 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQL:'
apache_beam.utils.subprocess_server: INFO: b'SELECT COUNT(*) AS `count`'
apache_beam.utils.subprocess_server: INFO: b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`'
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQLPlan>'
apache_beam.utils.subprocess_server: INFO: b'LogicalAggregate(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' LogicalProject($f0=[0])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:30 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b'BeamAggregationRel(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7ffaca5a3488> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2930-_4\n Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-171-_14\n Map(<lambda at sql_test.py:171>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_20\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7ffaca5a3b70> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2930-_4\n Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-171-_14\n Map(<lambda at sql_test.py:171>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_20\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 161, in test_zetasql_generate_data
assert_that(out, equal_to([(1, "foo", 3.14)]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0422061845-453999f7_2ba4cb13-aef8-473f-8702-ae6951069025 failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '53511']
root: DEBUG: Waiting for grpc channel to be ready at localhost:53511.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:53511'
root: DEBUG: Waiting for grpc channel to be ready at localhost:53511.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:35 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:53511.
root: DEBUG: Waiting for grpc channel to be ready at localhost:53511.
root: DEBUG: Waiting for grpc channel to be ready at localhost:53511.
root: DEBUG: Waiting for grpc channel to be ready at localhost:53511.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:36 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:37 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 6:18:41 AM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])"
apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7ffaca5a3488> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_6\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7ffaca5a3b70> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_6\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 92.010s
FAILED (errors=9)
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:3:job-server:sparkJobServerCleanup
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 18m 15s
180 actionable tasks: 126 executed, 50 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/txs4wmjwnwi3g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #134
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/134/display/redirect?page=changes>
Changes:
[sychen] Add a BQ option for configuring buffering duration when auto-sharding is
[suztomo] [BEAM-11010] Upgrading google-cloud-pubsublite to 0.13.2
[suztomo] [BEAM-11010] Copying SubscriberOptions from pubsublite repo
[suztomo] [BEAM-11010] Declaring flogger-system-backend to avoid conflicts
[Kyle Weaver] [BEAM-12194] Enable SqlTransform::registerUdaf in ZetaSQL.
[Kyle Weaver] [BEAM-12194] Code style changes from review.
[Boyuan Zhang] [BEAM-12114] Dataflow should apply KAFKA_READ_OVERRIDE when it's not
[noreply] Make sdk/worker_harness_container_image fully backwards compatible
------------------------------------------
[...truncated 1.54 MB...]
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 179, in test_windowing_before_sql
assert_that(out, equal_to([(1, ), (1, ), (1, )]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0422004307-e1c7eca7_8b635987-da65-414a-ab11-15bb7ac9bace failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '59997']
root: DEBUG: Waiting for grpc channel to be ready at localhost:59997.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:59997'
root: DEBUG: Waiting for grpc channel to be ready at localhost:59997.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:42:58 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:59997.
root: DEBUG: Waiting for grpc channel to be ready at localhost:59997.
root: DEBUG: Waiting for grpc channel to be ready at localhost:59997.
root: DEBUG: Waiting for grpc channel to be ready at localhost:59997.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:42:59 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:00 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:02 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQL:'
apache_beam.utils.subprocess_server: INFO: b'SELECT COUNT(*) AS `count`'
apache_beam.utils.subprocess_server: INFO: b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`'
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:02 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQLPlan>'
apache_beam.utils.subprocess_server: INFO: b'LogicalAggregate(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' LogicalProject($f0=[0])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:02 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b'BeamAggregationRel(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7f1902ada510> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7f1902adabf8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 164, in test_zetasql_generate_data
assert_that(out, equal_to([(1, "foo", 3.14)]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0422004318-b9f096d5_9008f10b-416c-4072-bdaa-3b7f85f3db3e failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '45343']
root: DEBUG: Waiting for grpc channel to be ready at localhost:45343.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:45343'
root: DEBUG: Waiting for grpc channel to be ready at localhost:45343.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:07 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:45343.
root: DEBUG: Waiting for grpc channel to be ready at localhost:45343.
root: DEBUG: Waiting for grpc channel to be ready at localhost:45343.
root: DEBUG: Waiting for grpc channel to be ready at localhost:45343.
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:08 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:09 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 22, 2021 12:43:13 AM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])"
apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7f1902ada510> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7f1902adabf8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 97.091s
FAILED (errors=9)
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 6499.
Stopping expansion service pid: 6502.
> Task :runners:spark:3:job-server:sparkJobServerCleanup
Stopping job server pid: 29088.
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 42m 47s
180 actionable tasks: 163 executed, 13 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/i7tzsqfu4g32m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #133
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/133/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12009] Copy CalcRelSplitter.
[Kyle Weaver] [BEAM-12009] Implement Calc splitting rule.
[Kyle Weaver] [BEAM-12009] Use different method signature to avoid spurious null check
[Kyle Weaver] [BEAM-12009] Move CalcRelSplitter to rel subpackage.
[Kyle Weaver] [BEAM-12009] Reduce code duplication between rules by re-implementing
[Kyle Weaver] [BEAM-12009] Only match in BeamCalcSplittingRule if ≥ 1 of its component
[anup.d] BEAM-12166:Beam Sql - Combine Accumulator return Map fails with class
[suztomo] [BEAM-8357] Upgrading auto-value to 1.8 from 1.7.4
[Kenneth Knowles] Build source release zip from RC tag
[aromanenko.dev] [BEAM-12197] TPC-DS: Fix SQL-queries syntax
------------------------------------------
[...truncated 1.53 MB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 179, in test_windowing_before_sql
assert_that(out, equal_to([(1, ), (1, ), (1, )]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421183646-87fea71f_491f62ae-532a-4376-9d47-048d09f78c2d failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '51677']
root: DEBUG: Waiting for grpc channel to be ready at localhost:51677.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:51677'
root: DEBUG: Waiting for grpc channel to be ready at localhost:51677.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:37 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:51677.
root: DEBUG: Waiting for grpc channel to be ready at localhost:51677.
root: DEBUG: Waiting for grpc channel to be ready at localhost:51677.
root: DEBUG: Waiting for grpc channel to be ready at localhost:51677.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:38 PM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:39 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQL:'
apache_beam.utils.subprocess_server: INFO: b'SELECT COUNT(*) AS `count`'
apache_beam.utils.subprocess_server: INFO: b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`'
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQLPlan>'
apache_beam.utils.subprocess_server: INFO: b'LogicalAggregate(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' LogicalProject($f0=[0])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:41 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b'BeamAggregationRel(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7f621dc84400> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7f621dc84ae8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 164, in test_zetasql_generate_data
assert_that(out, equal_to([(1, "foo", 3.14)]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421183657-1a3aeee8_02dede8e-f027-4813-a20f-f8ebbeb296de failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '58795']
root: DEBUG: Waiting for grpc channel to be ready at localhost:58795.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:58795'
root: DEBUG: Waiting for grpc channel to be ready at localhost:58795.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:46 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:58795.
root: DEBUG: Waiting for grpc channel to be ready at localhost:58795.
root: DEBUG: Waiting for grpc channel to be ready at localhost:58795.
root: DEBUG: Waiting for grpc channel to be ready at localhost:58795.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:47 PM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:48 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:36:52 PM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])"
apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7f621dc84400> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7f621dc84ae8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 94.979s
FAILED (errors=9)
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:3:job-server:sparkJobServerCleanup
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 30s
180 actionable tasks: 144 executed, 32 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/tgxq4khhk5isc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #132
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/132/display/redirect>
Changes:
------------------------------------------
[...truncated 999.30 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 179, in test_windowing_before_sql
assert_that(out, equal_to([(1, ), (1, ), (1, )]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421122118-aee0517a_abec5bf5-6996-43c7-bf09-8b836ad379b9 failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '42629']
root: DEBUG: Waiting for grpc channel to be ready at localhost:42629.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:42629'
root: DEBUG: Waiting for grpc channel to be ready at localhost:42629.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:08 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:42629.
root: DEBUG: Waiting for grpc channel to be ready at localhost:42629.
root: DEBUG: Waiting for grpc channel to be ready at localhost:42629.
root: DEBUG: Waiting for grpc channel to be ready at localhost:42629.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:09 PM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:10 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:12 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQL:'
apache_beam.utils.subprocess_server: INFO: b'SELECT COUNT(*) AS `count`'
apache_beam.utils.subprocess_server: INFO: b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`'
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:13 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQLPlan>'
apache_beam.utils.subprocess_server: INFO: b'LogicalAggregate(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' LogicalProject($f0=[0])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:13 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b'BeamAggregationRel(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7f9b96085400> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7f9b96085ae8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 164, in test_zetasql_generate_data
assert_that(out, equal_to([(1, "foo", 3.14)]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421122131-2dd9c7f_e9876924-0166-4286-ba66-6a6af792da4f failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '45995']
root: DEBUG: Waiting for grpc channel to be ready at localhost:45995.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:45995'
root: DEBUG: Waiting for grpc channel to be ready at localhost:45995.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:19 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:45995.
root: DEBUG: Waiting for grpc channel to be ready at localhost:45995.
root: DEBUG: Waiting for grpc channel to be ready at localhost:45995.
root: DEBUG: Waiting for grpc channel to be ready at localhost:45995.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:20 PM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:21 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:21:25 PM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])"
apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7f9b96085400> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7f9b96085ae8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 115.294s
FAILED (errors=9)
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:3:job-server:sparkJobServerCleanup
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 21m 0s
180 actionable tasks: 126 executed, 50 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/qhng5odqhemlc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #131
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/131/display/redirect?page=changes>
Changes:
[noreply] Update stringSlice to impl pkg.go.dev/flag#Getter.Get
[noreply] [BEAM-12174] Samza Portable Runner Support (#14554)
------------------------------------------
[...truncated 993.80 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 179, in test_windowing_before_sql
assert_that(out, equal_to([(1, ), (1, ), (1, )]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421063149-647fcffb_d6985fcf-3ae7-40d2-b1f9-cafc817cc2ed failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '59271']
root: DEBUG: Waiting for grpc channel to be ready at localhost:59271.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:59271'
root: DEBUG: Waiting for grpc channel to be ready at localhost:59271.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:41 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:59271.
root: DEBUG: Waiting for grpc channel to be ready at localhost:59271.
root: DEBUG: Waiting for grpc channel to be ready at localhost:59271.
root: DEBUG: Waiting for grpc channel to be ready at localhost:59271.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:42 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:43 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQL:'
apache_beam.utils.subprocess_server: INFO: b'SELECT COUNT(*) AS `count`'
apache_beam.utils.subprocess_server: INFO: b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`'
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:44 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQLPlan>'
apache_beam.utils.subprocess_server: INFO: b'LogicalAggregate(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' LogicalProject($f0=[0])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b'BeamAggregationRel(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7fa0364e6400> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7fa0364e6ae8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 164, in test_zetasql_generate_data
assert_that(out, equal_to([(1, "foo", 3.14)]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421063159-4ea18e0d_bb24d9f5-f9cd-4522-8c73-ea8d4bf44f88 failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '58517']
root: DEBUG: Waiting for grpc channel to be ready at localhost:58517.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:58517'
root: DEBUG: Waiting for grpc channel to be ready at localhost:58517.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:49 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:58517.
root: DEBUG: Waiting for grpc channel to be ready at localhost:58517.
root: DEBUG: Waiting for grpc channel to be ready at localhost:58517.
root: DEBUG: Waiting for grpc channel to be ready at localhost:58517.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:50 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:51 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 6:31:55 AM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])"
apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7fa0364e6400> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7fa0364e6ae8> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 85.108s
FAILED (errors=9)
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:3:job-server:sparkJobServerCleanup
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 45s
180 actionable tasks: 129 executed, 47 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/xdqzpe7xyhsha
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #130
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/130/display/redirect?page=changes>
Changes:
[kileysok] Update Java beam container image
[suztomo] Libraries-BOM 20.0.0 with gRPC and protobuf
[suztomo] Updating assertions for google-http-client 1.39.2
[suztomo] Declare gax-httpjson as a direct dependency of google-cloud-platform
[noreply] Add --sdk_container_image Python WorkerOption (#14557)
[noreply] [BEAM-2085] Add capability to declare resource hints in Python pipeline
------------------------------------------
[...truncated 1001.86 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 179, in test_windowing_before_sql
assert_that(out, equal_to([(1, ), (1, ), (1, )]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421004038-c6058ef7_c10ed6fc-76ca-405e-9f5e-0389adc0ab1b failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '34737']
root: DEBUG: Waiting for grpc channel to be ready at localhost:34737.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:34737'
root: DEBUG: Waiting for grpc channel to be ready at localhost:34737.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:28 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:34737.
root: DEBUG: Waiting for grpc channel to be ready at localhost:34737.
root: DEBUG: Waiting for grpc channel to be ready at localhost:34737.
root: DEBUG: Waiting for grpc channel to be ready at localhost:34737.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:29 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:30 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQL:'
apache_beam.utils.subprocess_server: INFO: b'SELECT COUNT(*) AS `count`'
apache_beam.utils.subprocess_server: INFO: b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`'
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:32 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQLPlan>'
apache_beam.utils.subprocess_server: INFO: b'LogicalAggregate(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' LogicalProject($f0=[0])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b'BeamAggregationRel(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7fcec0e42048> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7fcec0e42730> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 164, in test_zetasql_generate_data
assert_that(out, equal_to([(1, "foo", 3.14)]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 582, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0421004050-676bf771_8acf3909-0166-42cf-8012-3f5a6dc9a648 failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '43655']
root: DEBUG: Waiting for grpc channel to be ready at localhost:43655.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:43655'
root: DEBUG: Waiting for grpc channel to be ready at localhost:43655.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:39 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:43655.
root: DEBUG: Waiting for grpc channel to be ready at localhost:43655.
root: DEBUG: Waiting for grpc channel to be ready at localhost:43655.
root: DEBUG: Waiting for grpc channel to be ready at localhost:43655.
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:40 AM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:41 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 21, 2021 12:40:45 AM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])"
apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7fcec0e42048> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7fcec0e42730> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 97.788s
FAILED (errors=9)
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
> Task :runners:spark:3:job-server:sparkJobServerCleanup
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 33m 35s
180 actionable tasks: 138 executed, 38 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/r3ccx2peajp5a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark3 #129
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/129/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Populate runtime parameters in SDK worker startup.
[Robert Bradshaw] [BEAM-6597] Use runner capabilities to activate short ids.
[Robert Bradshaw] Fix wording.
[Robert Bradshaw] Comment about runtime value usage.
[noreply] Add tensorboard-data-server license URL for Python container image.
------------------------------------------
[...truncated 1.52 MB...]
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 179, in test_windowing_before_sql
assert_that(out, equal_to([(1, ), (1, ), (1, )]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 579, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0420190905-9befd09f_905b14e2-550e-463a-a063-8903f6576dbc failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '49335']
root: DEBUG: Waiting for grpc channel to be ready at localhost:49335.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:49335'
root: DEBUG: Waiting for grpc channel to be ready at localhost:49335.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49335.
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:08:54 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:49335.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49335.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49335.
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:08:55 PM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:08:56 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:08:59 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQL:'
apache_beam.utils.subprocess_server: INFO: b'SELECT COUNT(*) AS `count`'
apache_beam.utils.subprocess_server: INFO: b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`'
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:08:59 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: SQLPlan>'
apache_beam.utils.subprocess_server: INFO: b'LogicalAggregate(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' LogicalProject($f0=[0])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:08:59 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b'BeamAggregationRel(group=[{}], count=[COUNT()])'
apache_beam.utils.subprocess_server: INFO: b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7fa21ffe6378> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7fa21ffe6a60> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2955-_4\n Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-174-_14\n Map(<lambda at sql_test.py:174>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_20\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py",> line 164, in test_zetasql_generate_data
assert_that(out, equal_to([(1, "foo", 3.14)]))
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py",> line 579, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 112, in run
state = result.wait_until_finish()
File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 602, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline BeamApp-jenkins-0420190916-4eae49cb_ed679363-634c-4e53-b50b-0a454ec2824f failed in state FAILED: java.lang.NullPointerException
-------------------- >> begin captured logging << --------------------
apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar>
apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '49527']
root: DEBUG: Waiting for grpc channel to be ready at localhost:49527.
apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:49527'
root: DEBUG: Waiting for grpc channel to be ready at localhost:49527.
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:09:05 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms'
apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178'
apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a'
root: DEBUG: Waiting for grpc channel to be ready at localhost:49527.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49527.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49527.
root: DEBUG: Waiting for grpc channel to be ready at localhost:49527.
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:09:06 PM org.apache.beam.sdk.expansion.service.ExpansionService expand'
apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'"
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:09:07 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig'
apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach."
apache_beam.utils.subprocess_server: INFO: b'Apr 20, 2021 7:09:11 PM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal'
apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>'
apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])"
apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])'
apache_beam.utils.subprocess_server: INFO: b''
root: DEBUG: Sending SIGINT to job_server
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: No image given, using default Python SDK image
root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev
root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7fa21ffe6378> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7fa21ffe6a60> ====================
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2955-_6\n assert_that/Create/FlatMap(<lambda at core.py:2955>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'experiments' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_name' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'runner' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'temp_location' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'streaming' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_cache_millis' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'environment_options' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'job_endpoint' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'output_executable_path' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'sdk_worker_parallelism' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'dataflow_kms_key' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'enable_streaming_engine' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'project' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_region' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'worker_zone' was already added
apache_beam.runners.portability.portable_runner: DEBUG: Runner option 'zone' was already added
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STOPPED
apache_beam.runners.portability.portable_runner: INFO: Job state changed to STARTING
apache_beam.runners.portability.portable_runner: INFO: Job state changed to RUNNING
root: DEBUG: java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
root: ERROR: java.lang.NullPointerException
apache_beam.runners.portability.portable_runner: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 156.608s
FAILED (errors=9)
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 31216.
Stopping expansion service pid: 31219.
> Task :runners:spark:3:job-server:sparkJobServerCleanup
Stopping job server pid: 29027.
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 42m 57s
180 actionable tasks: 130 executed, 46 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/4zsoomdab4vi4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org