You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/11/15 05:32:14 UTC
Build failed in Jenkins: beam_PostCommit_Python2 #994
See <https://builds.apache.org/job/beam_PostCommit_Python2/994/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-8657] Combiner lifting fix for bundle-based direct runner.
------------------------------------------
[...truncated 94.87 KB...]
> Task :runners:java-fn-execution:compileJava FROM-CACHE
> Task :runners:java-fn-execution:classes UP-TO-DATE
> Task :runners:java-fn-execution:jar
> Task :runners:flink:1.9:compileJava FROM-CACHE
> Task :runners:flink:1.9:classes
> Task :runners:flink:1.9:jar
> Task :runners:flink:1.9:job-server:compileJava NO-SOURCE
> Task :runners:flink:1.9:job-server:classes UP-TO-DATE
> Task :runners:spark:compileJava FROM-CACHE
> Task :runners:spark:classes UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:worker:compileJava FROM-CACHE
> Task :runners:google-cloud-dataflow-java:worker:classes
> Task :runners:spark:jar
> Task :runners:spark:job-server:compileJava NO-SOURCE
> Task :runners:spark:job-server:classes UP-TO-DATE
> Task :sdks:python:test-suites:direct:py2:mongodbioIT
ccc272bb6e5bb74e35fd01e846b9f8cdceea9bcee35fec52867866a31fbe9540
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Obtaining file://<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python>
ERROR: Command errored out with exit status 1:
command: <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/bin/python2.7> -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/setup.py'"'"';> __file__='"'"'<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/setup.py'"'"';f=getattr(tokenize,> '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
cwd: <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/>
Complete output (27 lines):
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/setup.py>:198: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/setup.py",> line 264, in <module>
'test': generate_protos_first(test),
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/setuptools/__init__.py",> line 144, in setup
_install_setup_requires(attrs)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/setuptools/__init__.py",> line 139, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/setuptools/dist.py",> line 720, in fetch_build_eggs
replace_conflicting=True,
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/pkg_resources/__init__.py",> line 794, in resolve
new_requirements = dist.requires(req.extras)[::-1]
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/pkg_resources/__init__.py",> line 2736, in requires
dm = self._dep_map
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/pkg_resources/__init__.py",> line 3023, in _dep_map
self.__dep_map = self._compute_dependencies()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/pkg_resources/__init__.py",> line 3032, in _compute_dependencies
for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/pkg_resources/__init__.py",> line 3014, in _parsed_pkg_info
metadata = self.get_metadata(self.PKG_INFO)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/pkg_resources/__init__.py",> line 1420, in get_metadata
value = self._get(path)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/local/lib/python2.7/site-packages/pkg_resources/__init__.py",> line 1616, in _get
with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg/pytest_runner-5.2.dist-info/METADATA'>
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
> Task :sdks:python:test-suites:direct:py2:mongodbioIT FAILED
> Task :sdks:java:testing:expansion-service:buildTestExpansionServiceJar
> Task :runners:google-cloud-dataflow-java:worker:shadowJar
> Task :runners:flink:1.9:job-server:shadowJar
> Task :runners:flink:1.9:job-server-container:copyDockerfileDependencies
> Task :runners:flink:1.9:job-server-container:dockerPrepare
> Task :runners:flink:1.9:job-server-container:docker
Sending build context to Docker daemon 124.2MB
Step 1/7 : FROM openjdk:8
---> 57c2c2d2643d
Step 2/7 : MAINTAINER "Apache Beam <de...@beam.apache.org>"
> Task :runners:spark:job-server:shadowJar
> Task :runners:flink:1.9:job-server-container:docker
---> Running in 86d056212f70
Removing intermediate container 86d056212f70
---> 4ab873b68238
Step 3/7 : RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y libltdl7
---> Running in b5d9264b2f17
Get:1 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Ign:2 http://deb.debian.org/debian stretch InRelease
Get:3 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Get:5 http://deb.debian.org/debian stretch Release.gpg [2365 B]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [508 kB]
Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [27.9 kB]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7086 kB]
Fetched 7927 kB in 1s (4396 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
libltdl7
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 389 kB of archives.
After this operation, 427 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 libltdl7 amd64 2.4.6-2 [389 kB]
[91mdebconf: delaying package configuration, since apt-utils is not installed
[0mFetched 389 kB in 0s (5065 kB/s)
Selecting previously unselected package libltdl7:amd64.
(Reading database ...
(Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%(Reading database ... 12447 files and directories currently installed.)
Preparing to unpack .../libltdl7_2.4.6-2_amd64.deb ...
Unpacking libltdl7:amd64 (2.4.6-2) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Setting up libltdl7:amd64 (2.4.6-2) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Removing intermediate container b5d9264b2f17
---> c77410b71266
Step 4/7 : ADD beam-runners-flink-job-server.jar /opt/apache/beam/jars/
---> a708461a9c80
Step 5/7 : ADD flink-job-server.sh /opt/apache/beam/
---> c38d7796ca59
Step 6/7 : WORKDIR /opt/apache/beam
---> Running in 57591c0e5ec5
Removing intermediate container 57591c0e5ec5
---> e1a5e3e473cf
Step 7/7 : ENTRYPOINT ["./flink-job-server.sh"]
---> Running in fc5695315099
Removing intermediate container fc5695315099
---> 2045c5efab2e
Successfully built 2045c5efab2e
Successfully tagged jenkins-docker-apache.bintray.io/beam/flink-job-server:latest
> Task :sdks:go:resolveBuildDependencies
Resolving google.golang.org/api: commit='386d4e5f4f92f86e6aec85985761bba4b938a2d5', urls=[https://code.googlesource.com/google-api-go-client]
Resolving google.golang.org/genproto: commit='2b5a72b8730b0b16380010cfe5286c42108d88e7', urls=[https://github.com/google/go-genproto]
Resolving google.golang.org/grpc: commit='7646b5360d049a7ca31e9133315db43456f39e2e', urls=[https://github.com/grpc/grpc-go]
Resolving cached github.com/coreos/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/coreos/etcd.git, git@github.com:coreos/etcd.git]
Resolving cached github.com/coreos/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/coreos/etcd.git, git@github.com:coreos/etcd.git]
> Task :sdks:go:installDependencies
> Task :sdks:go:buildLinuxAmd64
> Task :sdks:go:goBuild
> Task :sdks:python:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/go>
> Task :sdks:java:container:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/go>
> Task :sdks:python:container:installDependencies
> Task :sdks:java:container:installDependencies
> Task :sdks:python:container:buildDarwinAmd64
> Task :sdks:java:container:buildLinuxAmd64
> Task :sdks:java:container:goBuild
> Task :sdks:java:container:dockerPrepare
> Task :sdks:python:container:buildLinuxAmd64
> Task :sdks:python:container:goBuild
> Task :sdks:python:container:py2:copyLauncherDependencies
> Task :sdks:java:container:docker
---> Running in 9457b4efe26b
Removing intermediate container 9457b4efe26b
---> cd4f6fa21a25
Step 3/9 : ADD target/slf4j-api.jar /opt/apache/beam/jars/
---> 083feb4233a1
Step 4/9 : ADD target/slf4j-jdk14.jar /opt/apache/beam/jars/
---> 17de27d52495
Step 5/9 : ADD target/beam-sdks-java-harness.jar /opt/apache/beam/jars/
---> 35dabacb52bd
Step 6/9 : ADD target/beam-sdks-java-io-kafka.jar /opt/apache/beam/jars/
---> f00ce7e4f78a
Step 7/9 : ADD target/kafka-clients.jar /opt/apache/beam/jars/
---> 673138607ac4
Step 8/9 : ADD target/linux_amd64/boot /opt/apache/beam/
---> 55e0e623cc1e
Step 9/9 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in cc2ddc2c82cd
Removing intermediate container cc2ddc2c82cd
---> b287f5717b0b
Successfully built b287f5717b0b
Successfully tagged apachebeam/java_sdk:2.18.0-SNAPSHOT
FAILURE: Build completed with 5 failures.
1: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/build.gradle'> line: 46
* What went wrong:
Execution failed for task ':sdks:python:sdist'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py2:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py2:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py2:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/direct/py2/build.gradle'> line: 107
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py2:mongodbioIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 18s
108 actionable tasks: 82 executed, 23 from cache, 3 up-to-date
Publishing build scan...
https://gradle.com/s/dync6foppybxk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python2 #997
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python2/997/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python2 #996
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python2/996/display/redirect?page=changes>
Changes:
[skoonce] [BEAM-8554] Use WorkItemCommitRequest protobuf fields to signal that a
------------------------------------------
[...truncated 1.61 MB...]
19/11/15 07:21:41 INFO BlockManager: Found block rdd_19_2 locally
19/11/15 07:21:41 INFO BlockManager: Found block rdd_19_1 locally
19/11/15 07:21:41 INFO BlockManager: Found block rdd_19_0 locally
19/11/15 07:21:41 INFO BlockManager: Found block rdd_19_3 locally
19/11/15 07:21:41 INFO Executor: Finished task 1.0 in stage 5.0 (TID 21). 10082 bytes result sent to driver
19/11/15 07:21:41 INFO TaskSetManager: Finished task 1.0 in stage 5.0 (TID 21) in 101 ms on localhost (executor driver) (1/4)
19/11/15 07:21:41 INFO Executor: Finished task 2.0 in stage 5.0 (TID 22). 10082 bytes result sent to driver
19/11/15 07:21:41 INFO TaskSetManager: Finished task 2.0 in stage 5.0 (TID 22) in 140 ms on localhost (executor driver) (2/4)
19/11/15 07:21:41 INFO Executor: Finished task 0.0 in stage 5.0 (TID 20). 10082 bytes result sent to driver
19/11/15 07:21:41 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 20) in 152 ms on localhost (executor driver) (3/4)
WARNING:apache_beam.io.filebasedsink:Deleting 4 existing files in target path matching: -*-of-%(num_shards)05d
19/11/15 07:22:03 INFO Executor: Finished task 3.0 in stage 5.0 (TID 23). 10125 bytes result sent to driver
19/11/15 07:22:03 INFO TaskSetManager: Finished task 3.0 in stage 5.0 (TID 23) in 22449 ms on localhost (executor driver) (4/4)
19/11/15 07:22:03 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool
19/11/15 07:22:03 INFO DAGScheduler: ResultStage 5 (collect at BoundedDataset.java:76) finished in 22.460 s
19/11/15 07:22:03 INFO DAGScheduler: Job 2 finished: collect at BoundedDataset.java:76, took 22.464917 s
19/11/15 07:22:03 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 176.0 B, free 13.5 GB)
19/11/15 07:22:03 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 702.0 B, free 13.5 GB)
19/11/15 07:22:03 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on localhost:35577 (size: 702.0 B, free: 13.5 GB)
19/11/15 07:22:03 INFO SparkContext: Created broadcast 11 from broadcast at SparkBatchPortablePipelineTranslator.java:336
19/11/15 07:22:03 INFO SparkPipelineRunner: Job BeamApp-jenkins-1115072135-93900ab9_bce16d1f-3174-48be-b6e7-cdb32ff41dce: Pipeline translated successfully. Computing outputs
19/11/15 07:22:03 INFO SparkContext: Starting job: foreach at BoundedDataset.java:124
19/11/15 07:22:03 INFO DAGScheduler: Got job 3 (foreach at BoundedDataset.java:124) with 4 output partitions
19/11/15 07:22:03 INFO DAGScheduler: Final stage: ResultStage 6 (foreach at BoundedDataset.java:124)
19/11/15 07:22:03 INFO DAGScheduler: Parents of final stage: List()
19/11/15 07:22:03 INFO DAGScheduler: Missing parents: List()
19/11/15 07:22:03 INFO DAGScheduler: Submitting ResultStage 6 (EmptyOutputSink_0 MapPartitionsRDD[38] at flatMap at SparkBatchPortablePipelineTranslator.java:311), which has no missing parents
19/11/15 07:22:03 INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 33.1 KB, free 13.5 GB)
19/11/15 07:22:03 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 12.8 KB, free 13.5 GB)
19/11/15 07:22:03 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on localhost:35577 (size: 12.8 KB, free: 13.5 GB)
19/11/15 07:22:03 INFO SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:1161
19/11/15 07:22:03 INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 6 (EmptyOutputSink_0 MapPartitionsRDD[38] at flatMap at SparkBatchPortablePipelineTranslator.java:311) (first 15 tasks are for partitions Vector(0, 1, 2, 3))
19/11/15 07:22:03 INFO TaskSchedulerImpl: Adding task set 6.0 with 4 tasks
19/11/15 07:22:03 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 24, localhost, executor driver, partition 0, PROCESS_LOCAL, 7868 bytes)
19/11/15 07:22:03 INFO TaskSetManager: Starting task 1.0 in stage 6.0 (TID 25, localhost, executor driver, partition 1, PROCESS_LOCAL, 7868 bytes)
19/11/15 07:22:03 INFO TaskSetManager: Starting task 2.0 in stage 6.0 (TID 26, localhost, executor driver, partition 2, PROCESS_LOCAL, 7868 bytes)
19/11/15 07:22:03 INFO TaskSetManager: Starting task 3.0 in stage 6.0 (TID 27, localhost, executor driver, partition 3, PROCESS_LOCAL, 7879 bytes)
19/11/15 07:22:03 INFO Executor: Running task 1.0 in stage 6.0 (TID 25)
19/11/15 07:22:03 INFO Executor: Running task 2.0 in stage 6.0 (TID 26)
19/11/15 07:22:03 INFO Executor: Running task 0.0 in stage 6.0 (TID 24)
19/11/15 07:22:03 INFO Executor: Running task 3.0 in stage 6.0 (TID 27)
19/11/15 07:22:03 INFO BlockManager: Found block rdd_19_3 locally
19/11/15 07:22:03 INFO BlockManager: Found block rdd_19_1 locally
19/11/15 07:22:03 INFO BlockManager: Found block rdd_19_0 locally
19/11/15 07:22:03 INFO BlockManager: Found block rdd_19_2 locally
19/11/15 07:22:03 INFO Executor: Finished task 1.0 in stage 6.0 (TID 25). 9384 bytes result sent to driver
19/11/15 07:22:03 INFO TaskSetManager: Finished task 1.0 in stage 6.0 (TID 25) in 69 ms on localhost (executor driver) (1/4)
19/11/15 07:22:04 INFO Executor: Finished task 0.0 in stage 6.0 (TID 24). 9341 bytes result sent to driver
19/11/15 07:22:04 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 24) in 88 ms on localhost (executor driver) (2/4)
19/11/15 07:22:04 INFO Executor: Finished task 2.0 in stage 6.0 (TID 26). 9341 bytes result sent to driver
19/11/15 07:22:04 INFO TaskSetManager: Finished task 2.0 in stage 6.0 (TID 26) in 104 ms on localhost (executor driver) (3/4)
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.12 seconds.
19/11/15 07:22:26 INFO Executor: Finished task 3.0 in stage 6.0 (TID 27). 9384 bytes result sent to driver
19/11/15 07:22:26 INFO TaskSetManager: Finished task 3.0 in stage 6.0 (TID 27) in 22248 ms on localhost (executor driver) (4/4)
19/11/15 07:22:26 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool
19/11/15 07:22:26 INFO DAGScheduler: ResultStage 6 (foreach at BoundedDataset.java:124) finished in 22.259 s
19/11/15 07:22:26 INFO DAGScheduler: Job 3 finished: foreach at BoundedDataset.java:124, took 22.263436 s
19/11/15 07:22:26 INFO SparkPipelineRunner: Job BeamApp-jenkins-1115072135-93900ab9_bce16d1f-3174-48be-b6e7-cdb32ff41dce finished.
19/11/15 07:22:26 INFO SparkUI: Stopped Spark web UI at http://localhost:4040
19/11/15 07:22:26 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/11/15 07:22:26 INFO MemoryStore: MemoryStore cleared
19/11/15 07:22:26 INFO BlockManager: BlockManager stopped
19/11/15 07:22:26 INFO BlockManagerMaster: BlockManagerMaster stopped
19/11/15 07:22:26 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
19/11/15 07:22:26 INFO SparkContext: Successfully stopped SparkContext
19/11/15 07:22:26 WARN SparkPipelineResult$BatchMode: Collecting monitoring infos is not implemented yet in Spark portable runner.
19/11/15 07:22:26 INFO AbstractArtifactRetrievalService: Manifest at /tmp/beam-tempL2N8Y5/artifactsnuZoj6/job_af827443-6285-46f7-8e2f-dc8fd1383fb3/MANIFEST has 1 artifact locations
19/11/15 07:22:26 INFO BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-tempL2N8Y5/artifactsnuZoj6/job_af827443-6285-46f7-8e2f-dc8fd1383fb3/
INFO:root:Job state changed to DONE
19/11/15 07:22:26 INFO InMemoryJobService: Getting job metrics for BeamApp-jenkins-1115072135-93900ab9_bce16d1f-3174-48be-b6e7-cdb32ff41dce
19/11/15 07:22:26 INFO InMemoryJobService: Finished getting job metrics for BeamApp-jenkins-1115072135-93900ab9_bce16d1f-3174-48be-b6e7-cdb32ff41dce
19/11/15 07:22:26 INFO ShutdownHookManager: Shutdown hook called
19/11/15 07:22:26 INFO ShutdownHookManager: Deleting directory /tmp/spark-aac3e6cb-60bd-4723-a0c7-d1845c345140
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 601, in pull_responses
for response in responses:
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573802546.684472919","description":"Error received from peer ipv4:127.0.0.1:44559","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 142, in run
for work_request in control_stub.Control(get_responses()):
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573802546.684495588","description":"Error received from peer ipv4:127.0.0.1:34085","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:root:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 271, in _read_inputs
for elements in elements_iterator:
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573802546.684449432","description":"Error received from peer ipv4:127.0.0.1:43493","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 285, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 271, in _read_inputs
for elements in elements_iterator:
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573802546.684449432","description":"Error received from peer ipv4:127.0.0.1:43493","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py2:postCommitPy2
> Task :sdks:python:test-suites:dataflow:py2:postCommitIT
test_leader_board_it (apache_beam.examples.complete.game.leader_board_it_test.LeaderBoardIT) ... ok
test_datastore_write_limit (apache_beam.io.gcp.datastore.v1new.datastore_write_it_test.DatastoreWriteIT) ... SKIP: GCP dependencies are not installed
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:723: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
test_game_stats_it (apache_beam.examples.complete.game.game_stats_it_test.GameStatsIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:797: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_user_score_it (apache_beam.examples.complete.game.user_score_it_test.UserScoreIT) ... ok
test_streaming_wordcount_it (apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:651: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1214: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1214: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:797: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
test_hourly_team_score_it (apache_beam.examples.complete.game.hourly_team_score_it_test.HourlyTeamScoreIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1211: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
test_avro_it (apache_beam.examples.fastavro_it_test.FastavroIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:723: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
test_bigquery_read_1M_python (apache_beam.io.gcp.bigquery_io_read_it_test.BigqueryIOReadIT) ... ok
test_multiple_destinations_transform (apache_beam.io.gcp.bigquery_test.BigQueryStreamingInsertTransformIntegrationTests) ... ok
test_copy (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_batch (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_batch_kms (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_batch_rewrite_token (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_kms (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_rewrite_token (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_value_provider_transform (apache_beam.io.gcp.bigquery_test.BigQueryStreamingInsertTransformIntegrationTests) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/fileio_test.py>:296: FutureWarning: MatchAll is experimental.
| 'GetPath' >> beam.Map(lambda metadata: metadata.path))
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/fileio_test.py>:307: FutureWarning: MatchAll is experimental.
| 'Checksums' >> beam.Map(compute_hash))
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/fileio_test.py>:307: FutureWarning: ReadMatches is experimental.
| 'Checksums' >> beam.Map(compute_hash))
test_big_query_read (apache_beam.io.gcp.bigquery_read_it_test.BigQueryReadIntegrationTests) ... ok
test_big_query_read_new_types (apache_beam.io.gcp.bigquery_read_it_test.BigQueryReadIntegrationTests) ... ok
test_bqfl_streaming (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... SKIP: TestStream is not supported on TestDataflowRunner
test_multiple_destinations_transform (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... ok
test_one_job_fails_all_jobs_fail (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... ok
test_transform_on_gcs (apache_beam.io.fileio_test.MatchIntegrationTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:73: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=kms_key))
test_file_loads (apache_beam.io.gcp.bigquery_test.PubSubBigQueryIT) ... SKIP: https://issuetracker.google.com/issues/118375066
test_streaming_inserts (apache_beam.io.gcp.bigquery_test.PubSubBigQueryIT) ... ok
test_parquetio_it (apache_beam.io.parquetio_it_test.TestParquetIT) ... ok
test_streaming_data_only (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_streaming_with_attributes (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_big_query_legacy_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_new_types (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_standard_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_standard_sql_kms_key_native (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_write (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_new_types (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_schema_autodetect (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... SKIP: DataflowRunner does not support schema autodetection
test_big_query_write_without_schema (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
Runs streaming Dataflow job and verifies that user metrics are reported ... ok
test_job_python_from_python_it (apache_beam.transforms.external_test_it.ExternalTransformIT) ... ok
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
test_metrics_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
test_datastore_write_limit (apache_beam.io.gcp.datastore_write_it_test.DatastoreWriteIT) ... ok
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 45 tests in 3448.731s
OK (SKIP=4)
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/direct/py2/build.gradle'> line: 48
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py2:directRunnerIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 58m 52s
119 actionable tasks: 95 executed, 21 from cache, 3 up-to-date
Publishing build scan...
https://gradle.com/s/dbnlun7rkdsey
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python2 #995
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python2/995/display/redirect>
Changes:
------------------------------------------
[...truncated 1.60 MB...]
19/11/15 06:18:18 INFO DAGScheduler: running: Set()
19/11/15 06:18:18 INFO DAGScheduler: waiting: Set(ResultStage 4)
19/11/15 06:18:18 INFO DAGScheduler: failed: Set()
19/11/15 06:18:18 INFO DAGScheduler: Submitting ResultStage 4 (MapPartitionsRDD[33] at map at BoundedDataset.java:75), which has no missing parents
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 23.1 KB, free 13.5 GB)
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 10.0 KB, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on localhost:43097 (size: 10.0 KB, free: 13.5 GB)
19/11/15 06:18:18 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:1161
19/11/15 06:18:18 INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 4 (MapPartitionsRDD[33] at map at BoundedDataset.java:75) (first 15 tasks are for partitions Vector(0, 1, 2, 3))
19/11/15 06:18:18 INFO TaskSchedulerImpl: Adding task set 4.0 with 4 tasks
19/11/15 06:18:18 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 16, localhost, executor driver, partition 0, NODE_LOCAL, 7662 bytes)
19/11/15 06:18:18 INFO TaskSetManager: Starting task 1.0 in stage 4.0 (TID 17, localhost, executor driver, partition 1, PROCESS_LOCAL, 7662 bytes)
19/11/15 06:18:18 INFO TaskSetManager: Starting task 2.0 in stage 4.0 (TID 18, localhost, executor driver, partition 2, PROCESS_LOCAL, 7662 bytes)
19/11/15 06:18:18 INFO TaskSetManager: Starting task 3.0 in stage 4.0 (TID 19, localhost, executor driver, partition 3, PROCESS_LOCAL, 7662 bytes)
19/11/15 06:18:18 INFO Executor: Running task 0.0 in stage 4.0 (TID 16)
19/11/15 06:18:18 INFO Executor: Running task 1.0 in stage 4.0 (TID 17)
19/11/15 06:18:18 INFO Executor: Running task 3.0 in stage 4.0 (TID 19)
19/11/15 06:18:18 INFO Executor: Running task 2.0 in stage 4.0 (TID 18)
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks including 0 local blocks and 0 remote blocks
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks including 0 local blocks and 0 remote blocks
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks including 0 local blocks and 0 remote blocks
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Getting 4 non-empty blocks including 4 local blocks and 0 remote blocks
19/11/15 06:18:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
19/11/15 06:18:18 INFO MemoryStore: Block rdd_32_1 stored as values in memory (estimated size 16.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added rdd_32_1 in memory on localhost:43097 (size: 16.0 B, free: 13.5 GB)
19/11/15 06:18:18 INFO Executor: Finished task 1.0 in stage 4.0 (TID 17). 10453 bytes result sent to driver
19/11/15 06:18:18 INFO TaskSetManager: Finished task 1.0 in stage 4.0 (TID 17) in 67 ms on localhost (executor driver) (1/4)
19/11/15 06:18:18 INFO MemoryStore: Block rdd_32_3 stored as values in memory (estimated size 16.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added rdd_32_3 in memory on localhost:43097 (size: 16.0 B, free: 13.5 GB)
19/11/15 06:18:18 INFO Executor: Finished task 3.0 in stage 4.0 (TID 19). 10453 bytes result sent to driver
19/11/15 06:18:18 INFO TaskSetManager: Finished task 3.0 in stage 4.0 (TID 19) in 87 ms on localhost (executor driver) (2/4)
19/11/15 06:18:18 INFO MemoryStore: Block rdd_32_2 stored as values in memory (estimated size 16.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added rdd_32_2 in memory on localhost:43097 (size: 16.0 B, free: 13.5 GB)
19/11/15 06:18:18 INFO Executor: Finished task 2.0 in stage 4.0 (TID 18). 10453 bytes result sent to driver
19/11/15 06:18:18 INFO TaskSetManager: Finished task 2.0 in stage 4.0 (TID 18) in 107 ms on localhost (executor driver) (3/4)
19/11/15 06:18:18 INFO MemoryStore: Block rdd_32_0 stored as values in memory (estimated size 920.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added rdd_32_0 in memory on localhost:43097 (size: 920.0 B, free: 13.5 GB)
19/11/15 06:18:18 INFO Executor: Finished task 0.0 in stage 4.0 (TID 16). 11065 bytes result sent to driver
19/11/15 06:18:18 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 16) in 116 ms on localhost (executor driver) (4/4)
19/11/15 06:18:18 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool
19/11/15 06:18:18 INFO DAGScheduler: ResultStage 4 (collect at BoundedDataset.java:76) finished in 0.123 s
19/11/15 06:18:18 INFO DAGScheduler: Job 1 finished: collect at BoundedDataset.java:76, took 1.136873 s
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 832.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 981.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on localhost:43097 (size: 981.0 B, free: 13.5 GB)
19/11/15 06:18:18 INFO SparkContext: Created broadcast 7 from broadcast at SparkBatchPortablePipelineTranslator.java:336
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 288.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 802.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on localhost:43097 (size: 802.0 B, free: 13.5 GB)
19/11/15 06:18:18 INFO SparkContext: Created broadcast 8 from broadcast at SparkBatchPortablePipelineTranslator.java:336
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 832.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 981.0 B, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on localhost:43097 (size: 981.0 B, free: 13.5 GB)
19/11/15 06:18:18 INFO SparkContext: Created broadcast 9 from broadcast at SparkBatchPortablePipelineTranslator.java:336
19/11/15 06:18:18 INFO SparkContext: Starting job: collect at BoundedDataset.java:76
19/11/15 06:18:18 INFO DAGScheduler: Got job 2 (collect at BoundedDataset.java:76) with 4 output partitions
19/11/15 06:18:18 INFO DAGScheduler: Final stage: ResultStage 5 (collect at BoundedDataset.java:76)
19/11/15 06:18:18 INFO DAGScheduler: Parents of final stage: List()
19/11/15 06:18:18 INFO DAGScheduler: Missing parents: List()
19/11/15 06:18:18 INFO DAGScheduler: Submitting ResultStage 5 (MapPartitionsRDD[36] at map at BoundedDataset.java:75), which has no missing parents
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 31.3 KB, free 13.5 GB)
19/11/15 06:18:18 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 12.3 KB, free 13.5 GB)
19/11/15 06:18:18 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on localhost:43097 (size: 12.3 KB, free: 13.5 GB)
19/11/15 06:18:18 INFO SparkContext: Created broadcast 10 from broadcast at DAGScheduler.scala:1161
19/11/15 06:18:18 INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 5 (MapPartitionsRDD[36] at map at BoundedDataset.java:75) (first 15 tasks are for partitions Vector(0, 1, 2, 3))
19/11/15 06:18:18 INFO TaskSchedulerImpl: Adding task set 5.0 with 4 tasks
19/11/15 06:18:18 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 20, localhost, executor driver, partition 0, PROCESS_LOCAL, 7868 bytes)
19/11/15 06:18:18 INFO TaskSetManager: Starting task 1.0 in stage 5.0 (TID 21, localhost, executor driver, partition 1, PROCESS_LOCAL, 7868 bytes)
19/11/15 06:18:18 INFO TaskSetManager: Starting task 2.0 in stage 5.0 (TID 22, localhost, executor driver, partition 2, PROCESS_LOCAL, 7868 bytes)
19/11/15 06:18:18 INFO TaskSetManager: Starting task 3.0 in stage 5.0 (TID 23, localhost, executor driver, partition 3, PROCESS_LOCAL, 7879 bytes)
19/11/15 06:18:18 INFO Executor: Running task 0.0 in stage 5.0 (TID 20)
19/11/15 06:18:18 INFO Executor: Running task 1.0 in stage 5.0 (TID 21)
19/11/15 06:18:18 INFO Executor: Running task 3.0 in stage 5.0 (TID 23)
19/11/15 06:18:18 INFO Executor: Running task 2.0 in stage 5.0 (TID 22)
19/11/15 06:18:18 INFO BlockManager: Found block rdd_19_1 locally
19/11/15 06:18:18 INFO BlockManager: Found block rdd_19_0 locally
19/11/15 06:18:18 INFO BlockManager: Found block rdd_19_3 locally
19/11/15 06:18:18 INFO BlockManager: Found block rdd_19_2 locally
19/11/15 06:18:18 INFO Executor: Finished task 0.0 in stage 5.0 (TID 20). 10082 bytes result sent to driver
19/11/15 06:18:18 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 20) in 73 ms on localhost (executor driver) (1/4)
19/11/15 06:18:18 INFO Executor: Finished task 2.0 in stage 5.0 (TID 22). 10082 bytes result sent to driver
19/11/15 06:18:18 INFO Executor: Finished task 1.0 in stage 5.0 (TID 21). 10082 bytes result sent to driver
19/11/15 06:18:18 INFO TaskSetManager: Finished task 2.0 in stage 5.0 (TID 22) in 135 ms on localhost (executor driver) (2/4)
19/11/15 06:18:18 INFO TaskSetManager: Finished task 1.0 in stage 5.0 (TID 21) in 135 ms on localhost (executor driver) (3/4)
WARNING:apache_beam.io.filebasedsink:Deleting 4 existing files in target path matching: -*-of-%(num_shards)05d
19/11/15 06:18:30 INFO Executor: Finished task 3.0 in stage 5.0 (TID 23). 10125 bytes result sent to driver
19/11/15 06:18:30 INFO TaskSetManager: Finished task 3.0 in stage 5.0 (TID 23) in 11929 ms on localhost (executor driver) (4/4)
19/11/15 06:18:30 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool
19/11/15 06:18:30 INFO DAGScheduler: ResultStage 5 (collect at BoundedDataset.java:76) finished in 11.936 s
19/11/15 06:18:30 INFO DAGScheduler: Job 2 finished: collect at BoundedDataset.java:76, took 11.939151 s
19/11/15 06:18:30 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 176.0 B, free 13.5 GB)
19/11/15 06:18:30 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 702.0 B, free 13.5 GB)
19/11/15 06:18:30 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on localhost:43097 (size: 702.0 B, free: 13.5 GB)
19/11/15 06:18:30 INFO SparkContext: Created broadcast 11 from broadcast at SparkBatchPortablePipelineTranslator.java:336
19/11/15 06:18:30 INFO SparkPipelineRunner: Job BeamApp-jenkins-1115061813-66173c84_850b2a90-c5a5-4712-97cd-2b13421f94c9: Pipeline translated successfully. Computing outputs
19/11/15 06:18:30 INFO SparkContext: Starting job: foreach at BoundedDataset.java:124
19/11/15 06:18:30 INFO DAGScheduler: Got job 3 (foreach at BoundedDataset.java:124) with 4 output partitions
19/11/15 06:18:30 INFO DAGScheduler: Final stage: ResultStage 6 (foreach at BoundedDataset.java:124)
19/11/15 06:18:30 INFO DAGScheduler: Parents of final stage: List()
19/11/15 06:18:30 INFO DAGScheduler: Missing parents: List()
19/11/15 06:18:30 INFO DAGScheduler: Submitting ResultStage 6 (EmptyOutputSink_0 MapPartitionsRDD[38] at flatMap at SparkBatchPortablePipelineTranslator.java:311), which has no missing parents
19/11/15 06:18:30 INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 33.1 KB, free 13.5 GB)
19/11/15 06:18:30 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 12.8 KB, free 13.5 GB)
19/11/15 06:18:30 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on localhost:43097 (size: 12.8 KB, free: 13.5 GB)
19/11/15 06:18:30 INFO SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:1161
19/11/15 06:18:30 INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 6 (EmptyOutputSink_0 MapPartitionsRDD[38] at flatMap at SparkBatchPortablePipelineTranslator.java:311) (first 15 tasks are for partitions Vector(0, 1, 2, 3))
19/11/15 06:18:30 INFO TaskSchedulerImpl: Adding task set 6.0 with 4 tasks
19/11/15 06:18:30 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 24, localhost, executor driver, partition 0, PROCESS_LOCAL, 7868 bytes)
19/11/15 06:18:30 INFO TaskSetManager: Starting task 1.0 in stage 6.0 (TID 25, localhost, executor driver, partition 1, PROCESS_LOCAL, 7868 bytes)
19/11/15 06:18:30 INFO TaskSetManager: Starting task 2.0 in stage 6.0 (TID 26, localhost, executor driver, partition 2, PROCESS_LOCAL, 7868 bytes)
19/11/15 06:18:30 INFO TaskSetManager: Starting task 3.0 in stage 6.0 (TID 27, localhost, executor driver, partition 3, PROCESS_LOCAL, 7879 bytes)
19/11/15 06:18:30 INFO Executor: Running task 1.0 in stage 6.0 (TID 25)
19/11/15 06:18:30 INFO Executor: Running task 3.0 in stage 6.0 (TID 27)
19/11/15 06:18:30 INFO Executor: Running task 2.0 in stage 6.0 (TID 26)
19/11/15 06:18:30 INFO Executor: Running task 0.0 in stage 6.0 (TID 24)
19/11/15 06:18:30 INFO BlockManager: Found block rdd_19_1 locally
19/11/15 06:18:30 INFO BlockManager: Found block rdd_19_2 locally
19/11/15 06:18:30 INFO BlockManager: Found block rdd_19_3 locally
19/11/15 06:18:30 INFO BlockManager: Found block rdd_19_0 locally
19/11/15 06:18:30 INFO Executor: Finished task 1.0 in stage 6.0 (TID 25). 9384 bytes result sent to driver
19/11/15 06:18:30 INFO TaskSetManager: Finished task 1.0 in stage 6.0 (TID 25) in 127 ms on localhost (executor driver) (1/4)
19/11/15 06:18:30 INFO Executor: Finished task 0.0 in stage 6.0 (TID 24). 9341 bytes result sent to driver
19/11/15 06:18:30 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 24) in 136 ms on localhost (executor driver) (2/4)
19/11/15 06:18:30 INFO Executor: Finished task 2.0 in stage 6.0 (TID 26). 9341 bytes result sent to driver
19/11/15 06:18:30 INFO TaskSetManager: Finished task 2.0 in stage 6.0 (TID 26) in 155 ms on localhost (executor driver) (3/4)
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.12 seconds.
19/11/15 06:18:42 INFO Executor: Finished task 3.0 in stage 6.0 (TID 27). 9384 bytes result sent to driver
19/11/15 06:18:42 INFO TaskSetManager: Finished task 3.0 in stage 6.0 (TID 27) in 12171 ms on localhost (executor driver) (4/4)
19/11/15 06:18:42 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool
19/11/15 06:18:42 INFO DAGScheduler: ResultStage 6 (foreach at BoundedDataset.java:124) finished in 12.179 s
19/11/15 06:18:42 INFO DAGScheduler: Job 3 finished: foreach at BoundedDataset.java:124, took 12.182197 s
19/11/15 06:18:42 INFO SparkPipelineRunner: Job BeamApp-jenkins-1115061813-66173c84_850b2a90-c5a5-4712-97cd-2b13421f94c9 finished.
19/11/15 06:18:42 INFO SparkUI: Stopped Spark web UI at http://localhost:4040
19/11/15 06:18:42 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/11/15 06:18:42 INFO MemoryStore: MemoryStore cleared
19/11/15 06:18:42 INFO BlockManager: BlockManager stopped
19/11/15 06:18:42 INFO BlockManagerMaster: BlockManagerMaster stopped
19/11/15 06:18:42 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
19/11/15 06:18:42 INFO SparkContext: Successfully stopped SparkContext
19/11/15 06:18:42 WARN SparkPipelineResult$BatchMode: Collecting monitoring infos is not implemented yet in Spark portable runner.
19/11/15 06:18:42 INFO AbstractArtifactRetrievalService: Manifest at /tmp/beam-tempor1DVO/artifacts0koDPC/job_e9628a09-d9e6-490e-afae-4b76d882d53c/MANIFEST has 1 artifact locations
19/11/15 06:18:42 INFO BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-tempor1DVO/artifacts0koDPC/job_e9628a09-d9e6-490e-afae-4b76d882d53c/
INFO:root:Job state changed to DONE
19/11/15 06:18:42 INFO InMemoryJobService: Getting job metrics for BeamApp-jenkins-1115061813-66173c84_850b2a90-c5a5-4712-97cd-2b13421f94c9
19/11/15 06:18:42 INFO InMemoryJobService: Finished getting job metrics for BeamApp-jenkins-1115061813-66173c84_850b2a90-c5a5-4712-97cd-2b13421f94c9
19/11/15 06:18:42 INFO ShutdownHookManager: Shutdown hook called
19/11/15 06:18:42 INFO ShutdownHookManager: Deleting directory /tmp/spark-9c2c4ac6-0c92-4ea5-b9ff-c94fd472b7bd
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 142, in run
for work_request in control_stub.Control(get_responses()):
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573798723.044874525","description":"Error received from peer ipv4:127.0.0.1:41779","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 601, in pull_responses
for response in responses:
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573798723.044831528","description":"Error received from peer ipv4:127.0.0.1:40047","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:root:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 271, in _read_inputs
for elements in elements_iterator:
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573798723.044839952","description":"Error received from peer ipv4:127.0.0.1:45865","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 285, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 271, in _read_inputs
for elements in elements_iterator:
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 395, in next
return self._next()
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py",> line 561, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1573798723.044839952","description":"Error received from peer ipv4:127.0.0.1:45865","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py2:postCommitPy2
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/dataflow/py2/build.gradle'> line: 85
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py2:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 29s
119 actionable tasks: 93 executed, 23 from cache, 3 up-to-date
Publishing build scan...
https://gradle.com/s/hnpzxemipsbhq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org