You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/09/11 18:18:52 UTC
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1319
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1319/display/redirect?page=changes>
Changes:
[noreply] [BEAM-10874] Support v2 Go protos. (#12816)
[noreply] [BEAM-10824] [BEAM-7654] Use deterministic hash functions in
[noreply] [BEAM-9561] Fix issue with pickling in doctests. (#12560)
[noreply] [BEAM-10009] Add beam:logical_type:micros_instant:v1 (#12764)
------------------------------------------
[...truncated 104.08 KB...]
Using cached azure_nspkg-3.0.2-py2-none-any.whl (3.0 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.15-py2.py3-none-any.whl (23 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached google_api_core-1.22.2-py2.py3-none-any.whl (91 kB)
Processing /home/jenkins/.cache/pip/wheels/de/3a/83/77a1e18e1a8757186df834b86ce6800120ac9c79cd8ca4091b/grpc_google_iam_v1-0.12.3-cp27-none-any.whl
Collecting google-resumable-media<2.0dev,>=0.5.0
Using cached google_resumable_media-1.0.0-py2.py3-none-any.whl (42 kB)
Collecting monotonic>=0.6; python_version == "2.7"
Using cached monotonic-1.5-py2.py3-none-any.whl (5.3 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (0.13.1)
Collecting packaging
Using cached packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting more-itertools<6.0.0,>=4.0.0; python_version <= "2.7"
Using cached more_itertools-5.0.0-py2-none-any.whl (52 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (1.9.0)
Collecting attrs>=17.4.0
Using cached attrs-20.2.0-py2.py3-none-any.whl (48 kB)
> Task :runners:direct-java:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :runners:direct-java:classes
> Task :sdks:java:container:pullLicenses
done.
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting beautifulsoup4<5.0,>=4.9.0
Using cached beautifulsoup4-4.9.1-py3-none-any.whl (115 kB)
Processing /home/jenkins/.cache/pip/wheels/c4/f0/ae/d4689c4532d1f111462ed6a884a7767d502e511ee65f0d8e1b/future-0.18.2-py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/57/d0/2c/e2003abb5bc1a94c2e8a6fe1c03b8055d074e34c13672e7eb7/PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
Using cached soupsieve-2.0.1-py3-none-any.whl (32 kB)
Collecting six>=1.9.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: soupsieve, beautifulsoup4, future, pyyaml, six, tenacity
Successfully installed beautifulsoup4-4.9.1 future-0.18.2 pyyaml-5.3.1 six-1.15.0 soupsieve-2.0.1 tenacity-5.1.5
> Task :runners:direct-java:shadowJar
> Task :sdks:python:installGcpTest
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: pathlib2>=2.2.0; python_version < "3.6" in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (2.3.5)
Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8" in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (1.7.0)
Collecting pytest-forked
Using cached pytest_forked-1.3.0-py2.py3-none-any.whl (4.7 kB)
Collecting execnet>=1.1
Using cached execnet-1.7.1-py2.py3-none-any.whl (39 kB)
Collecting docutils<0.16,>=0.10
Using cached docutils-0.15.2-py2-none-any.whl (548 kB)
Collecting ipaddress; python_version < "3"
Using cached ipaddress-1.0.23-py2.py3-none-any.whl (18 kB)
Collecting cffi!=1.11.3,>=1.8
Using cached cffi-1.14.2-cp27-cp27mu-manylinux1_x86_64.whl (388 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.0-py2.py3-none-any.whl (45 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB)
Collecting backports.functools-lru-cache>=1.2.1; python_version < "3.2"
Using cached backports.functools_lru_cache-1.6.1-py2.py3-none-any.whl (5.7 kB)
Requirement already satisfied: scandir; python_version < "3.5" in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from pathlib2>=2.2.0; python_version < "3.6"->pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (1.10.0)
Requirement already satisfied: contextlib2; python_version < "3" in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (0.6.0.post1)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (1.2.0)
Requirement already satisfied: configparser>=3.5; python_version < "3" in <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/build/gradleenv/1922375555/lib/python2.7/site-packages> (from importlib-metadata>=0.12; python_version < "3.8"->pytest<5.0,>=4.4.0->apache-beam==2.25.0.dev0) (4.0.2)
Collecting apipkg>=1.4
Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
> Task :sdks:java:testing:expansion-service:buildTestExpansionServiceJar
> Task :sdks:java:container:pullLicenses
Executing <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_dir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/java_third_party_licenses> --dep_url_yaml=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml>
INFO:root:Pulling license for 177 dependencies using 16 threads.
INFO:root:pull_licenses_java.py succeed. It took 2.508593 seconds with 16 threads.
Copy licenses to <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:python:installGcpTest
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.25.0.dev0-py2-none-any.whl size=2245787 sha256=041ffc99cb9b0a107b9f964eecaa7428eac6933b8018b04d066ea8c1dc819ac5
Stored in directory: /home/jenkins/.cache/pip/wheels/f2/b0/4f/c4389c81ddb5fa6a9261073a77dcdd1bd828e0d4b43417b55c
Successfully built apache-beam
> Task :runners:spark:compileJava
> Task :runners:portability:java:compileJava
> Task :sdks:java:container:copyDockerfileDependencies
> Task :sdks:python:installGcpTest
Installing collected packages: crcmod, dill, pytz, fastavro, docopt, certifi, urllib3, idna, chardet, requests, hdfs, httplib2, pbr, funcsigs, mock, numpy, pymongo, pyasn1, rsa, pyasn1-modules, oauth2client, pyparsing, pydot, python-dateutil, typing-extensions, avro, pyvcf, pyarrow, jmespath, docutils, botocore, s3transfer, boto3, azure-nspkg, azure-core, ipaddress, pycparser, cffi, cryptography, azure-storage-nspkg, oauthlib, requests-oauthlib, isodate, msrest, azure-storage-blob, cachetools, monotonic, fasteners, google-apitools, google-auth, googleapis-common-protos, google-api-core, google-cloud-core, google-cloud-datastore, grpc-google-iam-v1, google-cloud-pubsub, google-resumable-media, google-cloud-bigquery, google-cloud-bigtable, google-cloud-spanner, grpcio-gcp, google-cloud-dlp, google-cloud-language, google-cloud-videointelligence, google-cloud-vision, freezegun, nose, nose-xunitmp, parameterized, pyhamcrest, pyyaml, requests-mock, tenacity, atomicwrites, packaging, more-itertools, attrs, backports.functools-lru-cache, wcwidth, pytest, pytest-forked, apipkg, execnet, pytest-xdist, pytest-timeout, pandas, apache-beam
> Task :runners:spark:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :runners:spark:classes
> Task :runners:spark:jar
> Task :runners:spark:job-server:compileJava NO-SOURCE
> Task :runners:spark:job-server:classes UP-TO-DATE
> Task :sdks:java:extensions:sql:compileJava
> Task :runners:portability:java:classes
> Task :runners:portability:java:jar
> Task :sdks:java:core:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :sdks:java:core:testClasses
> Task :sdks:java:core:shadowTestJar
> Task :sdks:java:extensions:sql:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :runners:spark:job-server:shadowJar
> Task :sdks:java:extensions:sql:classes
> Task :sdks:java:extensions:sql:jar
> Task :sdks:java:extensions:sql:zetasql:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :sdks:java:extensions:sql:zetasql:classes
> Task :sdks:java:extensions:sql:zetasql:jar
> Task :runners:core-construction-java:compileTestJava
> Task :runners:core-java:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :runners:core-java:testClasses
> Task :runners:core-java:testJar
> Task :sdks:java:extensions:sql:expansion-service:compileJava
Note: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/src/main/java/org/apache/beam/sdk/extensions/sql/expansion/ExternalSqlTransformRegistrar.java> uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :sdks:java:extensions:sql:expansion-service:classes
> Task :runners:core-construction-java:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :runners:core-construction-java:testClasses
> Task :runners:core-construction-java:testJar
> Task :sdks:python:installGcpTest
Successfully installed apache-beam-2.25.0.dev0 apipkg-1.5 atomicwrites-1.4.0 attrs-20.2.0 avro-1.9.2 azure-core-1.8.1 azure-nspkg-3.0.2 azure-storage-blob-12.5.0 azure-storage-nspkg-3.1.0 backports.functools-lru-cache-1.6.1 boto3-1.14.59 botocore-1.17.59 cachetools-3.1.1 certifi-2020.6.20 cffi-1.14.2 chardet-3.0.4 crcmod-1.7 cryptography-3.1 dill-0.3.1.1 docopt-0.6.2 docutils-0.15.2 execnet-1.7.1 fastavro-0.23.6 fasteners-0.15 freezegun-0.3.15 funcsigs-1.0.2 google-api-core-1.22.2 google-apitools-0.5.31 google-auth-1.21.1 google-cloud-bigquery-1.27.2 google-cloud-bigtable-1.4.0 google-cloud-core-1.4.1 google-cloud-datastore-1.15.0 google-cloud-dlp-1.0.0 google-cloud-language-1.3.0 google-cloud-pubsub-1.7.0 google-cloud-spanner-1.19.0 google-cloud-videointelligence-1.15.0 google-cloud-vision-1.0.0 google-resumable-media-1.0.0 googleapis-common-protos-1.52.0 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 hdfs-2.5.8 httplib2-0.17.4 idna-2.10 ipaddress-1.0.23 isodate-0.6.0 jmespath-0.10.0 mock-2.0.0 monotonic-1.5 more-itertools-5.0.0 msrest-0.6.19 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.16.6 oauth2client-3.0.0 oauthlib-3.1.0 packaging-20.4 pandas-0.24.2 parameterized-0.7.4 pbr-5.5.0 pyarrow-0.16.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pydot-1.4.1 pyhamcrest-1.10.1 pymongo-3.11.0 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.3.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.1 pytz-2020.1 pyvcf-0.6.8 pyyaml-5.3.1 requests-2.24.0 requests-mock-1.8.0 requests-oauthlib-1.3.0 rsa-4.0 s3transfer-0.3.3 tenacity-5.1.5 typing-extensions-3.7.4.3 urllib3-1.25.10 wcwidth-0.2.5
> Task :runners:portability:java:compileTestJava
Note: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/portability/java/src/test/java/org/apache/beam/runners/portability/CloseableResourceTest.java> uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :runners:portability:java:testClasses
> Task :runners:portability:java:testJar
> Task :runners:spark:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :runners:spark:testClasses
> Task :runners:spark:testJar
> Task :sdks:java:extensions:sql:expansion-service:shadowJar
> Task :runners:spark:job-server:sparkJobServerSetup
Launching job server @ 18091 ...
> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/api: commit='386d4e5f4f92f86e6aec85985761bba4b938a2d5', urls=[https://code.googlesource.com/google-api-go-client]
Resolving google.golang.org/genproto: commit='2b5a72b8730b0b16380010cfe5286c42108d88e7', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='7646b5360d049a7ca31e9133315db43456f39e2e', urls=[https://github.com/grpc/grpc-go]
Resolving google.golang.org/protobuf: commit='d165be301fb1e13390ad453281ded24385fd8ebc', urls=[https://go.googlesource.com/protobuf]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, git@github.com:etcd-io/etcd.git]
> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild
> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go>
> Task :sdks:python:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go>
> Task :sdks:java:container:installDependencies
> Task :sdks:python:container:installDependencies
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:python:container:buildDarwinAmd64
> Task :sdks:python:container:buildLinuxAmd64
> Task :sdks:python:container:goBuild
> Task :sdks:python:container:py2:copyLauncherDependencies
> Task :sdks:python:container:py2:dockerPrepare
> Task :sdks:python:container:py2:docker
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 31285.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':release:go-licenses:java:dockerRun'.
> Process 'command 'docker'' finished with non-zero exit value 125
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 17m 55s
143 actionable tasks: 124 executed, 15 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/d7ub4q6t6o2vg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_XVR_Spark #1340
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1340/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_XVR_Spark - Build # 1339 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_XVR_Spark - Build # 1339 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1339/ to view the results.
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1338
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1338/display/redirect>
Changes:
------------------------------------------
[...truncated 320.38 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:00 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:00 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3eb50e2a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '59399']
DEBUG:root:Waiting for grpc channel to be ready at localhost:59399.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:59399
DEBUG:root:Waiting for grpc channel to be ready at localhost:59399.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:17 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:59399.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59399.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59399.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59399.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:18 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:19 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:21 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3eb50e2a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '43699']
DEBUG:root:Waiting for grpc channel to be ready at localhost:43699.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:43699
DEBUG:root:Waiting for grpc channel to be ready at localhost:43699.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:39 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:43699.
DEBUG:root:Waiting for grpc channel to be ready at localhost:43699.
DEBUG:root:Waiting for grpc channel to be ready at localhost:43699.
DEBUG:root:Waiting for grpc channel to be ready at localhost:43699.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:40 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:41 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:43 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:43 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:27:43 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3eb50e2a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '46239']
DEBUG:root:Waiting for grpc channel to be ready at localhost:46239.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:46239
DEBUG:root:Waiting for grpc channel to be ready at localhost:46239.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:28:00 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:46239.
DEBUG:root:Waiting for grpc channel to be ready at localhost:46239.
DEBUG:root:Waiting for grpc channel to be ready at localhost:46239.
DEBUG:root:Waiting for grpc channel to be ready at localhost:46239.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:28:01 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:28:02 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3eb50e2a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 187.616s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 55s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mzwidv42oadt4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1337
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1337/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-6928] Update changelog with WriteToBigQuery changed requirements.
------------------------------------------
[...truncated 347.26 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:36 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb922465a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '57081']
DEBUG:root:Waiting for grpc channel to be ready at localhost:57081.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:57081
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:53 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:57081.
DEBUG:root:Waiting for grpc channel to be ready at localhost:57081.
DEBUG:root:Waiting for grpc channel to be ready at localhost:57081.
DEBUG:root:Waiting for grpc channel to be ready at localhost:57081.
DEBUG:root:Waiting for grpc channel to be ready at localhost:57081.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:54 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:55 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:56 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:56 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:26:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb922465a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '36831']
DEBUG:root:Waiting for grpc channel to be ready at localhost:36831.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:36831
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:14 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:36831.
DEBUG:root:Waiting for grpc channel to be ready at localhost:36831.
DEBUG:root:Waiting for grpc channel to be ready at localhost:36831.
DEBUG:root:Waiting for grpc channel to be ready at localhost:36831.
DEBUG:root:Waiting for grpc channel to be ready at localhost:36831.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:15 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:16 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb922465a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '37697']
DEBUG:root:Waiting for grpc channel to be ready at localhost:37697.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:37697
DEBUG:root:Waiting for grpc channel to be ready at localhost:37697.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:34 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:37697.
DEBUG:root:Waiting for grpc channel to be ready at localhost:37697.
DEBUG:root:Waiting for grpc channel to be ready at localhost:37697.
DEBUG:root:Waiting for grpc channel to be ready at localhost:37697.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:35 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 6:27:36 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb922465a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 185.093s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 32s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5dm46ke7gxc4m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1336
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1336/display/redirect?page=changes>
Changes:
[txian] Support NUMERIC in spanner schema parser
[Kyle Weaver] [BEAM-9575] Only copy the Spark runner jar, not whatever other jars
[noreply] [BEAM-10641] Add eliminate_common_key_with_none graph optimizer (#12787)
------------------------------------------
[...truncated 828.56 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:28:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:28:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0b21022a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '45699']
DEBUG:root:Waiting for grpc channel to be ready at localhost:45699.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:45699
DEBUG:root:Waiting for grpc channel to be ready at localhost:45699.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:11 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:45699.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45699.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45699.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45699.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:12 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:13 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:15 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0b21022a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '47401']
DEBUG:root:Waiting for grpc channel to be ready at localhost:47401.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:47401
DEBUG:root:Waiting for grpc channel to be ready at localhost:47401.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:33 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:47401.
DEBUG:root:Waiting for grpc channel to be ready at localhost:47401.
DEBUG:root:Waiting for grpc channel to be ready at localhost:47401.
DEBUG:root:Waiting for grpc channel to be ready at localhost:47401.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:34 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:35 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:37 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0b21022a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '44077']
DEBUG:root:Waiting for grpc channel to be ready at localhost:44077.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:44077
DEBUG:root:Waiting for grpc channel to be ready at localhost:44077.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:54 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:44077.
DEBUG:root:Waiting for grpc channel to be ready at localhost:44077.
DEBUG:root:Waiting for grpc channel to be ready at localhost:44077.
DEBUG:root:Waiting for grpc channel to be ready at localhost:44077.
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:55 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 16, 2020 12:29:56 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0b21022a28> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 195.657s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 50s
153 actionable tasks: 117 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7bkthog6s562y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1335
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1335/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Use index for partitioning of elementwise operations with multiple
[Robert Bradshaw] [BEAM-10873] Introduce partitioning session for stronger testing.
[Robert Bradshaw] [BEAM-10873] Use partitioinig session for tests.
[noreply] Merge pull request #12704 from [BEAM-10603] Implement the new Large
[noreply] write to file ability for java Nexmark suite (#12813)
[noreply] add readme file to python nexmark suites (#12808)
[noreply] Merge pull request #12807 from [BEAM-2855] implement query 10
[noreply] Merge pull request #12770 from [BEAM-10545] Assembled the extension with
[noreply] Document GroupBy transform. (#12834)
[noreply] [BEAM-10886] Fix Java Wordcount Direct Runner (windows-latest) (#12846)
[noreply] * [BEAM-10705] Extract and use the filename when downloading a remote
------------------------------------------
[...truncated 867.48 KB...]
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:31:47 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:31:47 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9b232c9938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '59703']
DEBUG:root:Waiting for grpc channel to be ready at localhost:59703.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:59703
DEBUG:root:Waiting for grpc channel to be ready at localhost:59703.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:05 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:59703.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59703.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59703.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59703.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:06 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:07 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:09 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9b232c9938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '49857']
DEBUG:root:Waiting for grpc channel to be ready at localhost:49857.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:49857
DEBUG:root:Waiting for grpc channel to be ready at localhost:49857.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:28 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:49857.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49857.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49857.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49857.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:29 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:30 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:31 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9b232c9938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '38971']
DEBUG:root:Waiting for grpc channel to be ready at localhost:38971.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:38971
DEBUG:root:Waiting for grpc channel to be ready at localhost:38971.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:49 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:38971.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38971.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38971.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38971.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:50 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:32:51 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9b232c9938> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 206.227s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 7744.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 53s
153 actionable tasks: 121 executed, 28 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jykrcqjfbplh4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1334
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1334/display/redirect>
Changes:
------------------------------------------
[...truncated 345.82 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:27:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:27:51 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9ef0024de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '45893']
DEBUG:root:Waiting for grpc channel to be ready at localhost:45893.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:45893
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:07 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:45893.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45893.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45893.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45893.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45893.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:09 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:09 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:11 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9ef0024de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '45581']
DEBUG:root:Waiting for grpc channel to be ready at localhost:45581.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:45581
DEBUG:root:Waiting for grpc channel to be ready at localhost:45581.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:29 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:45581.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45581.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45581.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45581.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:30 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:31 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:32 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9ef0024de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '41705']
DEBUG:root:Waiting for grpc channel to be ready at localhost:41705.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:41705
DEBUG:root:Waiting for grpc channel to be ready at localhost:41705.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:49 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:41705.
DEBUG:root:Waiting for grpc channel to be ready at localhost:41705.
DEBUG:root:Waiting for grpc channel to be ready at localhost:41705.
DEBUG:root:Waiting for grpc channel to be ready at localhost:41705.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:50 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:51 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9ef0024de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 183.769s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 28m 45s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ao77sjii43gl4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1333
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1333/display/redirect?page=changes>
Changes:
[noreply] [BEAM-10781] Add
------------------------------------------
[...truncated 321.90 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:25:58 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:25:58 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe096835de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '39209']
DEBUG:root:Waiting for grpc channel to be ready at localhost:39209.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:39209
DEBUG:root:Waiting for grpc channel to be ready at localhost:39209.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:15 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:39209.
DEBUG:root:Waiting for grpc channel to be ready at localhost:39209.
DEBUG:root:Waiting for grpc channel to be ready at localhost:39209.
DEBUG:root:Waiting for grpc channel to be ready at localhost:39209.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:16 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:17 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe096835de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '52625']
DEBUG:root:Waiting for grpc channel to be ready at localhost:52625.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:52625
DEBUG:root:Waiting for grpc channel to be ready at localhost:52625.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:36 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:52625.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52625.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52625.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52625.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:37 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:38 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:40 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe096835de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '42273']
DEBUG:root:Waiting for grpc channel to be ready at localhost:42273.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:42273
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:56 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:42273.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42273.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42273.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42273.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42273.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:57 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 6:26:58 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe096835de8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 181.958s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 54s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/rsrsl72hzu5us
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1332
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1332/display/redirect?page=changes>
Changes:
[Boyuan Zhang] [BEAM-10505][BEAM-10530] Add truncate capability.
[sychen] Add max buffering duration to GroupIntoBatches
[noreply] [BEAM-10252] Add null check on logical type override (#12831)
[noreply] Updates Dataflow containers used by unreleased SDKs. (#12833)
[noreply] [BEAM-9615] Add initial schema proto documentation. (#12553)
------------------------------------------
[...truncated 864.04 KB...]
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:27:46 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:27:46 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff676652d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '38441']
DEBUG:root:Waiting for grpc channel to be ready at localhost:38441.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:38441
DEBUG:root:Waiting for grpc channel to be ready at localhost:38441.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:03 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:38441.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38441.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38441.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38441.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:04 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:04 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:06 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:06 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:06 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff676652d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '45871']
DEBUG:root:Waiting for grpc channel to be ready at localhost:45871.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:45871
DEBUG:root:Waiting for grpc channel to be ready at localhost:45871.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:24 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:45871.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45871.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45871.
DEBUG:root:Waiting for grpc channel to be ready at localhost:45871.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:25 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:26 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:27 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff676652d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '47491']
DEBUG:root:Waiting for grpc channel to be ready at localhost:47491.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:47491
DEBUG:root:Waiting for grpc channel to be ready at localhost:47491.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:44 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:47491.
DEBUG:root:Waiting for grpc channel to be ready at localhost:47491.
DEBUG:root:Waiting for grpc channel to be ready at localhost:47491.
DEBUG:root:Waiting for grpc channel to be ready at localhost:47491.
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:45 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 15, 2020 12:28:46 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff676652d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 183.683s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 26248.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 28m 26s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6swodrxhfx3t6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1331
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1331/display/redirect?page=changes>
Changes:
[qinyeli] Removing dead code from Interactive Beam.
[pulasthi911] adding twister2 quickstart
[pulasthi911] adding twister2 documentaion
[noreply] [BEAM-10463] Fix minor typos
[nosacky] [BEAM-10886] Also publish build scans from Github actions.
------------------------------------------
[...truncated 316.24 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:29 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7199e78d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '40547']
DEBUG:root:Waiting for grpc channel to be ready at localhost:40547.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:40547
DEBUG:root:Waiting for grpc channel to be ready at localhost:40547.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:46 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:40547.
DEBUG:root:Waiting for grpc channel to be ready at localhost:40547.
DEBUG:root:Waiting for grpc channel to be ready at localhost:40547.
DEBUG:root:Waiting for grpc channel to be ready at localhost:40547.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:47 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:48 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:49 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:32:50 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7199e78d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '52257']
DEBUG:root:Waiting for grpc channel to be ready at localhost:52257.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:52257
DEBUG:root:Waiting for grpc channel to be ready at localhost:52257.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:08 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:52257.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52257.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52257.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52257.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:09 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:10 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:12 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:12 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:12 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7199e78d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '52507']
DEBUG:root:Waiting for grpc channel to be ready at localhost:52507.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:52507
DEBUG:root:Waiting for grpc channel to be ready at localhost:52507.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:30 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:52507.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52507.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52507.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52507.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:31 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:33:32 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7199e78d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 193.947s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 33m 24s
153 actionable tasks: 117 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/hpfj6ose7abm4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1330
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1330/display/redirect>
Changes:
------------------------------------------
[...truncated 838.97 KB...]
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:14 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:14 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa455338d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '52347']
DEBUG:root:Waiting for grpc channel to be ready at localhost:52347.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:52347
DEBUG:root:Waiting for grpc channel to be ready at localhost:52347.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:31 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:52347.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52347.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52347.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52347.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:32 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:33 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:35 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa455338d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '53141']
DEBUG:root:Waiting for grpc channel to be ready at localhost:53141.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:53141
DEBUG:root:Waiting for grpc channel to be ready at localhost:53141.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:52 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:53141.
DEBUG:root:Waiting for grpc channel to be ready at localhost:53141.
DEBUG:root:Waiting for grpc channel to be ready at localhost:53141.
DEBUG:root:Waiting for grpc channel to be ready at localhost:53141.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:53 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:54 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:48:56 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa455338d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '52931']
DEBUG:root:Waiting for grpc channel to be ready at localhost:52931.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:52931
DEBUG:root:Waiting for grpc channel to be ready at localhost:52931.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:49:13 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:52931.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52931.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52931.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52931.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:49:14 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:49:15 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa455338d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 195.138s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 29104.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 48m 37s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iwdhxmtrsks3s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1329
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1329/display/redirect>
Changes:
------------------------------------------
[...truncated 863.90 KB...]
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:14 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ffdd09d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '54601']
DEBUG:root:Waiting for grpc channel to be ready at localhost:54601.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:54601
DEBUG:root:Waiting for grpc channel to be ready at localhost:54601.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:30 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:54601.
DEBUG:root:Waiting for grpc channel to be ready at localhost:54601.
DEBUG:root:Waiting for grpc channel to be ready at localhost:54601.
DEBUG:root:Waiting for grpc channel to be ready at localhost:54601.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:31 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:32 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ffdd09d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '56467']
DEBUG:root:Waiting for grpc channel to be ready at localhost:56467.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:56467
DEBUG:root:Waiting for grpc channel to be ready at localhost:56467.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:51 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:56467.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56467.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56467.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56467.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:52 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:52 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:28:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ffdd09d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '50619']
DEBUG:root:Waiting for grpc channel to be ready at localhost:50619.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:50619
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:29:11 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:50619.
DEBUG:root:Waiting for grpc channel to be ready at localhost:50619.
DEBUG:root:Waiting for grpc channel to be ready at localhost:50619.
DEBUG:root:Waiting for grpc channel to be ready at localhost:50619.
DEBUG:root:Waiting for grpc channel to be ready at localhost:50619.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:29:12 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 6:29:13 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ffdd09d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 184.274s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 32704.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 5s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/krljojywk3q5k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1328
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1328/display/redirect>
Changes:
------------------------------------------
[...truncated 347.42 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:26:47 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:26:47 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0c538c9d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '58167']
DEBUG:root:Waiting for grpc channel to be ready at localhost:58167.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:58167
DEBUG:root:Waiting for grpc channel to be ready at localhost:58167.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:03 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:58167.
DEBUG:root:Waiting for grpc channel to be ready at localhost:58167.
DEBUG:root:Waiting for grpc channel to be ready at localhost:58167.
DEBUG:root:Waiting for grpc channel to be ready at localhost:58167.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:04 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:05 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:07 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:07 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:07 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0c538c9d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '55433']
DEBUG:root:Waiting for grpc channel to be ready at localhost:55433.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:55433
DEBUG:root:Waiting for grpc channel to be ready at localhost:55433.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:25 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:55433.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55433.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55433.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55433.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:26 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:27 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:28 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0c538c9d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '55577']
DEBUG:root:Waiting for grpc channel to be ready at localhost:55577.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:55577
DEBUG:root:Waiting for grpc channel to be ready at localhost:55577.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:45 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:55577.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55577.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55577.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55577.
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:46 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 14, 2020 12:27:47 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0c538c9d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 184.159s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 40s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3jaa4y6tvrbl4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1327
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1327/display/redirect>
Changes:
------------------------------------------
[...truncated 341.10 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:19 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f58d875cd70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '58949']
DEBUG:root:Waiting for grpc channel to be ready at localhost:58949.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:58949
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:36 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:58949.
DEBUG:root:Waiting for grpc channel to be ready at localhost:58949.
DEBUG:root:Waiting for grpc channel to be ready at localhost:58949.
DEBUG:root:Waiting for grpc channel to be ready at localhost:58949.
DEBUG:root:Waiting for grpc channel to be ready at localhost:58949.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:37 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:38 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:39 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:39 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:39 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f58d875cd70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '56469']
DEBUG:root:Waiting for grpc channel to be ready at localhost:56469.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:56469
DEBUG:root:Waiting for grpc channel to be ready at localhost:56469.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:57 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:56469.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56469.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56469.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56469.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:58 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:30:59 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:31:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:31:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:31:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f58d875cd70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '56403']
DEBUG:root:Waiting for grpc channel to be ready at localhost:56403.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:56403
DEBUG:root:Waiting for grpc channel to be ready at localhost:56403.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:31:18 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:56403.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56403.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56403.
DEBUG:root:Waiting for grpc channel to be ready at localhost:56403.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:31:19 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:31:20 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f58d875cd70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 189.302s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 14s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/r5kou25bpm3y4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1326
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1326/display/redirect>
Changes:
------------------------------------------
[...truncated 322.65 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7febf4f94d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '38495']
DEBUG:root:Waiting for grpc channel to be ready at localhost:38495.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:38495
DEBUG:root:Waiting for grpc channel to be ready at localhost:38495.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:37 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:38495.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38495.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38495.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38495.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:38 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:39 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7febf4f94d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '49785']
DEBUG:root:Waiting for grpc channel to be ready at localhost:49785.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:49785
DEBUG:root:Waiting for grpc channel to be ready at localhost:49785.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:58 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:49785.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49785.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49785.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49785.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:26:59 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:27:00 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:27:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:27:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:27:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7febf4f94d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '54493']
DEBUG:root:Waiting for grpc channel to be ready at localhost:54493.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:54493
DEBUG:root:Waiting for grpc channel to be ready at localhost:54493.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:27:18 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:54493.
DEBUG:root:Waiting for grpc channel to be ready at localhost:54493.
DEBUG:root:Waiting for grpc channel to be ready at localhost:54493.
DEBUG:root:Waiting for grpc channel to be ready at localhost:54493.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:27:19 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:27:20 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7febf4f94d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 185.254s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 15s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/pnefvbz7jtz7w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1325
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1325/display/redirect>
Changes:
------------------------------------------
[...truncated 317.44 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:13 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb0445e2d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '57767']
DEBUG:root:Waiting for grpc channel to be ready at localhost:57767.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:57767
DEBUG:root:Waiting for grpc channel to be ready at localhost:57767.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:29 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:57767.
DEBUG:root:Waiting for grpc channel to be ready at localhost:57767.
DEBUG:root:Waiting for grpc channel to be ready at localhost:57767.
DEBUG:root:Waiting for grpc channel to be ready at localhost:57767.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:30 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:31 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:33 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb0445e2d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '52749']
DEBUG:root:Waiting for grpc channel to be ready at localhost:52749.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:52749
DEBUG:root:Waiting for grpc channel to be ready at localhost:52749.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:51 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:52749.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52749.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52749.
DEBUG:root:Waiting for grpc channel to be ready at localhost:52749.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:52 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:53 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:54 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:26:55 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb0445e2d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '38083']
DEBUG:root:Waiting for grpc channel to be ready at localhost:38083.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:38083
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:27:11 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:38083.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38083.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38083.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38083.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38083.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:27:12 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 6:27:13 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb0445e2d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 184.348s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 7s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/46o7k662rdcve
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1324
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1324/display/redirect?page=changes>
Changes:
[noreply] [BEAM-10861] Adds URNs and payloads to PubSub transforms to allow
------------------------------------------
[...truncated 364.47 KB...]
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:28:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:28:45 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa6af5b4d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '48969']
DEBUG:root:Waiting for grpc channel to be ready at localhost:48969.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:48969
DEBUG:root:Waiting for grpc channel to be ready at localhost:48969.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:02 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:48969.
DEBUG:root:Waiting for grpc channel to be ready at localhost:48969.
DEBUG:root:Waiting for grpc channel to be ready at localhost:48969.
DEBUG:root:Waiting for grpc channel to be ready at localhost:48969.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:03 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:04 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:06 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:06 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:06 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa6af5b4d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '38193']
DEBUG:root:Waiting for grpc channel to be ready at localhost:38193.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:38193
DEBUG:root:Waiting for grpc channel to be ready at localhost:38193.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:24 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:38193.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38193.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38193.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38193.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:25 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:26 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:27 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:27 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:27 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa6af5b4d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '42833']
DEBUG:root:Waiting for grpc channel to be ready at localhost:42833.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:42833
DEBUG:root:Waiting for grpc channel to be ready at localhost:42833.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:45 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:42833.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42833.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42833.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42833.
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:46 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 13, 2020 12:29:46 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa6af5b4d70> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 187.324s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 31576.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 40s
153 actionable tasks: 140 executed, 9 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5angofecbsnh2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1323
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1323/display/redirect>
Changes:
------------------------------------------
[...truncated 839.67 KB...]
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:20 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd3f3b59aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '43661']
DEBUG:root:Waiting for grpc channel to be ready at localhost:43661.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:43661
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:36 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:43661.
DEBUG:root:Waiting for grpc channel to be ready at localhost:43661.
DEBUG:root:Waiting for grpc channel to be ready at localhost:43661.
DEBUG:root:Waiting for grpc channel to be ready at localhost:43661.
DEBUG:root:Waiting for grpc channel to be ready at localhost:43661.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:38 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:38 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:40 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd3f3b59aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '55751']
DEBUG:root:Waiting for grpc channel to be ready at localhost:55751.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:55751
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:57 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:55751.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55751.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55751.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55751.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55751.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:58 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:28:59 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:29:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:29:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:29:01 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd3f3b59aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '42291']
DEBUG:root:Waiting for grpc channel to be ready at localhost:42291.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:42291
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:29:17 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
DEBUG:root:Waiting for grpc channel to be ready at localhost:42291.
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:42291.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42291.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42291.
DEBUG:root:Waiting for grpc channel to be ready at localhost:42291.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:29:18 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:29:19 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd3f3b59aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 180.349s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 8227.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 15s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mze6llzrcngjy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1322
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1322/display/redirect>
Changes:
------------------------------------------
[...truncated 327.93 KB...]
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:33 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:33 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ac9db9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '60651']
DEBUG:root:Waiting for grpc channel to be ready at localhost:60651.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:60651
DEBUG:root:Waiting for grpc channel to be ready at localhost:60651.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:50 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:60651.
DEBUG:root:Waiting for grpc channel to be ready at localhost:60651.
DEBUG:root:Waiting for grpc channel to be ready at localhost:60651.
DEBUG:root:Waiting for grpc channel to be ready at localhost:60651.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:51 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:52 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:28:53 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ac9db9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '46133']
DEBUG:root:Waiting for grpc channel to be ready at localhost:46133.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:46133
DEBUG:root:Waiting for grpc channel to be ready at localhost:46133.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:11 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:46133.
DEBUG:root:Waiting for grpc channel to be ready at localhost:46133.
DEBUG:root:Waiting for grpc channel to be ready at localhost:46133.
DEBUG:root:Waiting for grpc channel to be ready at localhost:46133.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:12 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:13 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:14 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:14 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:14 PM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ac9db9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '41295']
DEBUG:root:Waiting for grpc channel to be ready at localhost:41295.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:41295
DEBUG:root:Waiting for grpc channel to be ready at localhost:41295.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:31 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:41295.
DEBUG:root:Waiting for grpc channel to be ready at localhost:41295.
DEBUG:root:Waiting for grpc channel to be ready at localhost:41295.
DEBUG:root:Waiting for grpc channel to be ready at localhost:41295.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:32 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:29:33 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ac9db9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 193.278s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
Stopping job server pid: 32583.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 26s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3o3q7mf7z7hti
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1321
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1321/display/redirect>
Changes:
------------------------------------------
[...truncated 832.16 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:25:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:25:57 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7efdc954caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '55113']
DEBUG:root:Waiting for grpc channel to be ready at localhost:55113.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:55113
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:14 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:55113.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55113.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55113.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55113.
DEBUG:root:Waiting for grpc channel to be ready at localhost:55113.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:15 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:16 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:17 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:18 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7efdc954caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '59663']
DEBUG:root:Waiting for grpc channel to be ready at localhost:59663.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:59663
DEBUG:root:Waiting for grpc channel to be ready at localhost:59663.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:35 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:59663.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59663.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59663.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59663.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:36 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:37 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:38 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:39 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7efdc954caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '53685']
DEBUG:root:Waiting for grpc channel to be ready at localhost:53685.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:53685
DEBUG:root:Waiting for grpc channel to be ready at localhost:53685.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:56 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:53685.
DEBUG:root:Waiting for grpc channel to be ready at localhost:53685.
DEBUG:root:Waiting for grpc channel to be ready at localhost:53685.
DEBUG:root:Waiting for grpc channel to be ready at localhost:53685.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:57 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 6:26:57 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7efdc954caa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 186.286s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 52s
153 actionable tasks: 115 executed, 34 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/v4ybh6z4f4f6c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Spark #1320
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/1320/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-10670] Add support for bundle finalization to DoFnOperator for
[Luke Cwik] [BEAM-10670] Make Flink to be opt-out for SplittableDoFn powering the
[Luke Cwik] fixup! Fix translation issue where we should never be translating
[Luke Cwik] fixup! Address PR comments
[Robert Bradshaw] [BEAM-10833] Fix type inference for BUILD_MAP.
[Robert Bradshaw] Guard test for Python 3.
[Robert Bradshaw] [BEAM-9547] A couple trivial but common dataframe methods.
[Robert Bradshaw] [BEAM-9561] Improve WontImplement reporting.
[Alan Myrvold] [BEAM-10868] Fix build (docker run) concurrency issue when building
[noreply] [BEAM-10876] Fix TypeError in dataflow_metrics when distribution sum …
[noreply] [BEAM-7523] Enable KafkaCSVTableIT using KafkaContainer (#12826)
[noreply] Deprecate obsolete CombineFn.add_inputs. (#12802)
[noreply] [BEAM-10678] Split up assertion for clarity (#12828)
------------------------------------------
[...truncated 318.29 KB...]
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `PCOLLECTION`.`a` * `PCOLLECTION`.`a` AS `s`, `LENGTH`(`PCOLLECTION`.`b`) AS `c`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:03 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(s=[*($0, $0)], c=[LENGTH($1)])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:03 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..1=[{inputs}], expr#2=[*($t0, $t0)], expr#3=[LENGTH($t1)], s=[$t2], c=[$t3])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa8d23d9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:23 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:150>)_14\n Map(<lambda at sql_test.py:150>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_2/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_6SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_18/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_18\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_19\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_21\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_23\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_25\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_26\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_27\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_28\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_29\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_30\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_31\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_tagged_join (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '49183']
DEBUG:root:Waiting for grpc channel to be ready at localhost:49183.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:49183
DEBUG:root:Waiting for grpc channel to be ready at localhost:49183.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:20 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:49183.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49183.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49183.
DEBUG:root:Waiting for grpc channel to be ready at localhost:49183.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:21 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:22 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT `simple`.`id` AS `id`, `enrich`.`metadata` AS `metadata`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`simple` AS `simple`
INFO:apache_beam.utils.subprocess_server:INNER JOIN `beam`.`enrich` AS `enrich` ON `simple`.`id` = `enrich`.`id`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalProject(id=[$0], metadata=[$4])
INFO:apache_beam.utils.subprocess_server: LogicalJoin(condition=[=($0, $3)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:24 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamCalcRel(expr#0..4=[{inputs}], id=[$t2], metadata=[$t1])
INFO:apache_beam.utils.subprocess_server: BeamCoGBKJoinRel(condition=[=($2, $0)], joinType=[inner])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, enrich]])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, simple]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa8d23d9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:42 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create enrich/Impulse_3\n Create enrich/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/FlatMap(<lambda at core.py:2876>)_4\n Create enrich/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create enrich/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create enrich/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create enrich/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create enrich/Map(decode)_13\n Create enrich/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Impulse_15\n Create simple/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/FlatMap(<lambda at core.py:2876>)_16\n Create simple/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys_19\n Create simple/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_21\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_22\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_23\n Create simple/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys_24\n Create simple/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create simple/Map(decode)_25\n Create simple/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_6/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_5/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/left_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/right_TimestampCombiner/Flatten.PCollections:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeylhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/extractKeyrhs/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/GBK:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Join.Impl/CoGroup.ExpandCrossProduct/ParDo(ConvertCoGbkResult)/ParMultiDo(ConvertCoGbkResult):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select)\n SqlTransform(beam:external:java:sql:v1)/BeamCoGBKJoinRel_95/Select.Fields/ParDo(Select)/ParMultiDo(Select):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_7SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamCalcRel_96/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_29\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_30\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_32\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_33\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_34\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_36\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_37\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_38\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_39\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_40\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_41\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_42\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '38189']
DEBUG:root:Waiting for grpc channel to be ready at localhost:38189.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:38189
DEBUG:root:Waiting for grpc channel to be ready at localhost:38189.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:42 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:38189.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38189.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38189.
DEBUG:root:Waiting for grpc channel to be ready at localhost:38189.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:43 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:44 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:46 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQL:
INFO:apache_beam.utils.subprocess_server:SELECT COUNT(*) AS `count`
INFO:apache_beam.utils.subprocess_server:FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:46 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: SQLPlan>
INFO:apache_beam.utils.subprocess_server:LogicalAggregate(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: LogicalProject($f0=[0])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:32:46 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO:apache_beam.utils.subprocess_server:INFO: BEAMPlan>
INFO:apache_beam.utils.subprocess_server:BeamAggregationRel(group=[{}], count=[COUNT()])
INFO:apache_beam.utils.subprocess_server: BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO:apache_beam.utils.subprocess_server:
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa8d23d9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:29 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create/Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2876>)_4\n Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create/Map(decode)_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map(<lambda at sql_test.py:172>)_14\n Map(<lambda at sql_test.py:172>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto(WindowIntoFn)_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamIOSourceRel_3/Convert.ConvertTransform/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/toRow/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_20\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar>
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.25.0-SNAPSHOT.jar'> '59419']
DEBUG:root:Waiting for grpc channel to be ready at localhost:59419.
INFO:apache_beam.utils.subprocess_server:Starting expansion service at localhost:59419
DEBUG:root:Waiting for grpc channel to be ready at localhost:59419.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:33:03 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms
INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server: beam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@626b2d4a
INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$1/2121744517@5e91993f
DEBUG:root:Waiting for grpc channel to be ready at localhost:59419.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59419.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59419.
DEBUG:root:Waiting for grpc channel to be ready at localhost:59419.
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:33:04 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'
INFO:apache_beam.utils.subprocess_server:Sep 12, 2020 12:33:05 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach.
DEBUG:root:Sending SIGINT to job_server
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.25.0.dev. If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa8d23d9aa0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/MapElements/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper)\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc)\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at core.py:2876>)_6\n assert_that/Create/FlatMap(<lambda at core.py:2876>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Create/Map(decode)_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that/Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_name' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'runner' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'temp_location' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'streaming' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'environment_cache_millis' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'files_to_stage' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'job_endpoint' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'output_executable_path' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'sdk_worker_parallelism' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'dataflow_kms_key' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'enable_streaming_engine' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'project' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_region' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'worker_zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'zone' was already added
DEBUG:apache_beam.runners.portability.portable_runner:Runner option 'experiments' was already added
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
----------------------------------------------------------------------
XML: nosetests-xlangSqlValidateRunner.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 208.525s
OK
> Task :runners:spark:job-server:sparkJobServerCleanup
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:spark:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/runners/spark/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 32m 59s
153 actionable tasks: 126 executed, 23 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nkgj5n4tqvcea
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org