You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/08/08 17:46:45 UTC
Build failed in Jenkins: beam_PostCommit_Python2 #169
See <https://builds.apache.org/job/beam_PostCommit_Python2/169/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-7912] Optimize GroupIntoBatches for batch Dataflow pipelines.
------------------------------------------
[...truncated 651.81 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 11 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error response from daemon: Cannot kill container: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b: No such container: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:179)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:134)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:174)
... 21 more
[flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Could not restart the job BeamApp-root-0808165523-6df9f59e (e41d9049b2757e60dad1a221eb21f468) because the restart strategy prevented it.
java.lang.Exception: The user defined 'open()' method caused an exception: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:211)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:202)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:185)
at org.apache.beam.runners.flink.translation.functions.FlinkDefaultExecutableStageContext.getStageBundleFactory(FlinkDefaultExecutableStageContext.java:49)
at org.apache.beam.runners.flink.translation.functions.ReferenceCountingFlinkExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingFlinkExecutableStageContextFactory.java:203)
at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.open(FlinkExecutableStageFunction.java:129)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:494)
... 3 more
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:179)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:160)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:178)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:162)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 11 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error response from daemon: Cannot kill container: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b: No such container: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:179)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:134)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:174)
... 21 more
[flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Job e41d9049b2757e60dad1a221eb21f468 reached globally terminal state FAILED.
[flink-runner-job-invoker] INFO org.apache.flink.runtime.minicluster.MiniCluster - Shutting down Flink Mini Cluster
[flink-runner-job-invoker] INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shutting down rest endpoint.
[flink-akka.actor.default-dispatcher-16] INFO org.apache.flink.runtime.jobmaster.JobMaster - Stopping the JobMaster for job BeamApp-root-0808165523-6df9f59e(e41d9049b2757e60dad1a221eb21f468).
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Stopping TaskExecutor akka://flink/user/taskmanager_0.
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager - Shutting down TaskExecutorLocalStateStoresManager.
[flink-akka.actor.default-dispatcher-16] INFO org.apache.flink.runtime.jobmaster.JobMaster - Close ResourceManager connection c570f48c89d752aac5de996906312ec5: JobManager is shutting down..
[flink-akka.actor.default-dispatcher-14] INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPool - Suspending SlotPool.
[flink-akka.actor.default-dispatcher-14] INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPool - Stopping SlotPool.
[flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping dispatcher akka://flink/user/dispatcher62a5d009-3db7-412c-b205-e990106f858e.
[flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping all currently running jobs of dispatcher akka://flink/user/dispatcher62a5d009-3db7-412c-b205-e990106f858e.
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager removed spill file directory /tmp/flink-io-950d3d57-4881-47ed-a828-d85b93339827
[flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - JobManager for job e41d9049b2757e60dad1a221eb21f468 with leader id acbbe1be17c8376133942f3fe2f24eb1 lost leadership.
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.io.network.NetworkEnvironment - Shutting down the network environment and its components.
[flink-akka.actor.default-dispatcher-14] INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Disconnect job manager acbbe1be17c8376133942f3fe2f24eb1@akka://flink/user/jobmanager_1 for job e41d9049b2757e60dad1a221eb21f468 from the resource manager.
[flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager - Closing the SlotManager.
[flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager - Suspending the SlotManager.
[flink-akka.actor.default-dispatcher-16] INFO org.apache.flink.runtime.rest.handler.legacy.backpressure.StackTraceSampleCoordinator - Shutting down stack trace sample coordinator.
[flink-akka.actor.default-dispatcher-16] INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopped dispatcher akka://flink/user/dispatcher62a5d009-3db7-412c-b205-e990106f858e.
[ForkJoinPool.commonPool-worker-11] INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Removing cache directory /tmp/flink-web-ui
[ForkJoinPool.commonPool-worker-11] INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shut down complete.
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Stop job leader service.
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Stopped TaskExecutor akka://flink/user/taskmanager_0.
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping Akka RPC service.
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.blob.PermanentBlobCache - Shutting down BLOB cache
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.blob.TransientBlobCache - Shutting down BLOB cache
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.blob.BlobServer - Stopped BLOB server at 0.0.0.0:38221
[flink-akka.actor.default-dispatcher-17] INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped Akka RPC service.
[flink-runner-job-invoker] ERROR org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation - Error during job invocation BeamApp-root-0808165523-6df9f59e_9278d37d-b323-44d1-887d-f7115fb5b4ab.
org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:632)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:234)
at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
at org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator$BatchTranslationContext.execute(FlinkBatchPortablePipelineTranslator.java:200)
at org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:87)
at org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:63)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:74)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Exception: The user defined 'open()' method caused an exception: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
... 1 more
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:211)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:202)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:185)
at org.apache.beam.runners.flink.translation.functions.FlinkDefaultExecutableStageContext.getStageBundleFactory(FlinkDefaultExecutableStageContext.java:49)
at org.apache.beam.runners.flink.translation.functions.ReferenceCountingFlinkExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingFlinkExecutableStageContextFactory.java:203)
at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.open(FlinkExecutableStageFunction.java:129)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:494)
... 3 more
Caused by: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:179)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:160)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:178)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:162)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 11 more
Suppressed: java.io.IOException: Received exit code 1 for command 'docker kill 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error response from daemon: Cannot kill container: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b: No such container: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:179)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:134)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:174)
... 21 more
[flink-runner-job-invoker] INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job_32cc5ce1-52d2-4717-ad48-b307cf653de0/MANIFEST has 1 artifact locations
[flink-runner-job-invoker] INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService - Removed dir /tmp/beam-artifact-staging/job_32cc5ce1-52d2-4717-ad48-b307cf653de0/
ERROR:root:java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/transforms/external_test.py",> line 236, in <module>
ExternalTransformTest.run_pipeline_with_portable_runner(pipeline_options)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/transforms/external_test.py",> line 186, in run_pipeline_with_portable_runner
pipeline_options, ExternalTransformTest.expansion_service_port, True)
File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/transforms/external_test.py",> line 219, in run_pipeline
result = p.run()
File "apache_beam/testing/test_pipeline.py", line 109, in run
state = result.wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 446, in wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline BeamApp-root-0808165523-6df9f59e_9278d37d-b323-44d1-887d-f7115fb5b4ab failed in state FAILED: java.io.IOException: Received exit code 1 for command 'docker inspect -f {{.State.Running}} 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b'. stderr: Error: No such object: 5b50b301553d233e574908b6711bc4fe29adb69be2c4fc5d154ccf461122ed0b
> Task :sdks:python:test-suites:portable:py2:crossLanguagePythonJavaFlink FAILED
> Task :sdks:python:test-suites:dataflow:py2:postCommitIT
test_datastore_wordcount_it (apache_beam.examples.cookbook.datastore_wordcount_it_test.DatastoreWordCountIT) ... ok
test_leader_board_it (apache_beam.examples.complete.game.leader_board_it_test.LeaderBoardIT) ... ok
test_datastore_write_limit (apache_beam.io.gcp.datastore.v1new.datastore_write_it_test.DatastoreWriteIT) ... SKIP: GCP dependencies are not installed
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:695: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
test_game_stats_it (apache_beam.examples.complete.game.game_stats_it_test.GameStatsIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:642: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_streaming_wordcount_it (apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1145: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
test_hourly_team_score_it (apache_beam.examples.complete.game.hourly_team_score_it_test.HourlyTeamScoreIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1145: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:642: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
test_avro_it (apache_beam.examples.fastavro_it_test.FastavroIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:695: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=transform.kms_key))
test_user_score_it (apache_beam.examples.complete.game.user_score_it_test.UserScoreIT) ... ok
test_bigquery_read_1M_python (apache_beam.io.gcp.bigquery_io_read_it_test.BigqueryIOReadIT) ... ok
test_copy (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_batch (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_batch_kms (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_batch_rewrite_token (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_kms (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
test_copy_rewrite_token (apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/fileio_test.py>:232: FutureWarning: MatchAll is experimental.
| 'GetPath' >> beam.Map(lambda metadata: metadata.path))
test_multiple_destinations_transform (apache_beam.io.gcp.bigquery_test.BigQueryStreamingInsertTransformIntegrationTests) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: MatchAll is experimental.
| 'Checksums' >> beam.Map(compute_hash))
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/fileio_test.py>:243: FutureWarning: ReadMatches is experimental.
| 'Checksums' >> beam.Map(compute_hash))
test_value_provider_transform (apache_beam.io.gcp.bigquery_test.BigQueryStreamingInsertTransformIntegrationTests) ... ok
test_big_query_read (apache_beam.io.gcp.bigquery_read_it_test.BigQueryReadIntegrationTests) ... ok
test_big_query_read_new_types (apache_beam.io.gcp.bigquery_read_it_test.BigQueryReadIntegrationTests) ... ok
test_transform_on_gcs (apache_beam.io.fileio_test.MatchIntegrationTest) ... ok
test_bqfl_streaming (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... SKIP: TestStream is not supported on TestDataflowRunner
test_multiple_destinations_transform (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... ok
test_one_job_fails_all_jobs_fail (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) ... ok
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:73: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=kms_key))
test_parquetio_it (apache_beam.io.parquetio_it_test.TestParquetIT) ... ok
test_job_python_from_python_it (apache_beam.transforms.external_test_it.ExternalTransformIT) ... ok
test_streaming_data_only (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_streaming_with_attributes (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_big_query_legacy_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_new_types (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_standard_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_standard_sql_kms_key_native (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_write (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_new_types (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_schema_autodetect (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... SKIP: DataflowRunner does not support schema autodetection
test_big_query_write_without_schema (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
test_metrics_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
test_datastore_write_limit (apache_beam.io.gcp.datastore_write_it_test.DatastoreWriteIT) ... ok
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 42 tests in 3535.340s
OK (SKIP=3)
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/portable/py2/build.gradle'> line: 87
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py2:crossLanguagePythonJavaFlink'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 0m 25s
111 actionable tasks: 91 executed, 17 from cache, 3 up-to-date
Publishing build scan...
https://gradle.com/s/eqmwbu3ijjwju
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python2 #171
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python2/171/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python2 #170
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python2/170/display/redirect?page=changes>
Changes:
[heejong] [BEAM-7924] Failure in Python 2 postcommit: crossLanguagePythonJavaFlink
[david.moravek] [BEAM-7613] HadoopFileSystem can work with more than one cluster.
------------------------------------------
[...truncated 319.27 KB...]
Verified fast protobuf used.
Removing intermediate container 31c6e318e7b2
---> 4b008a2b2cc2
Step 6/9 : COPY target/apache-beam.tar.gz /opt/apache/beam/tars/
---> e093e5502246
Step 7/9 : RUN pip install /opt/apache/beam/tars/apache-beam.tar.gz[gcp] && rm -rf /root/.cache/pip
---> Running in 74e65362ac57
> Task :sdks:python:test-suites:direct:py2:hdfsIntegrationTest
Stopping hdfs_it-jenkins-beam_postcommit_python2-170_namenode_1 ... done
Aborting on container exit...
real 1m41.311s
user 0m1.175s
sys 0m0.160s
+ finally
+ docker-compose -p hdfs_IT-jenkins-beam_PostCommit_Python2-170 --no-ansi down
Removing hdfs_it-jenkins-beam_postcommit_python2-170_test_1 ...
Removing hdfs_it-jenkins-beam_postcommit_python2-170_datanode_1 ...
Removing hdfs_it-jenkins-beam_postcommit_python2-170_namenode_1 ...
Removing hdfs_it-jenkins-beam_postcommit_python2-170_datanode_1 ... done
Removing hdfs_it-jenkins-beam_postcommit_python2-170_namenode_1 ... done
Removing hdfs_it-jenkins-beam_postcommit_python2-170_test_1 ... done
Removing network hdfs_it-jenkins-beam_postcommit_python2-170_test_net
real 0m1.085s
user 0m0.631s
sys 0m0.101s
> Task :sdks:python:container:docker
[91mDEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
[0mProcessing /opt/apache/beam/tars/apache-beam.tar.gz
> Task :sdks:python:test-suites:direct:py2:mongodbioIT
Using default tag: latest
latest: Pulling from library/mongo
Digest: sha256:395609bdf0f9514b4fc220d1f681948566cc42a70f619776a5f61afc60340138
Status: Image is up to date for mongo:latest
e936fb53c037836f62e167fa75a97e6290405989f0e8cbc7f2bd1f6a2d439b22
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Obtaining file://<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python>
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (1.7)
Requirement already satisfied: dill<0.2.10,>=0.2.9 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.2.9)
Requirement already satisfied: fastavro<0.22,>=0.21.4 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.21.24)
Requirement already satisfied: future<1.0.0,>=0.16.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.16.0)
Requirement already satisfied: grpcio<2,>=1.8 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (1.22.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (2.5.8)
Requirement already satisfied: httplib2<=0.12.0,>=0.8 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.12.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (2.0.0)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (3.8.0)
Requirement already satisfied: oauth2client<4,>=2.0.1 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (3.0.0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (3.9.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (1.4.1)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (2.8.0)
Requirement already satisfied: pytz>=2018.3 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (2019.2)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (3.13)
Requirement already satisfied: avro<2.0.0,>=1.8.1 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (1.9.0)
Requirement already satisfied: futures<4.0.0,>=3.2.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (3.3.0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.6.8)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (3.6.6)
Requirement already satisfied: pyarrow<0.15.0,>=0.11.1 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.14.1)
Requirement already satisfied: nose>=1.3.7 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (1.3.7)
Requirement already satisfied: nose_xunitmp>=0.4.1 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.4.1)
Requirement already satisfied: numpy<2,>=1.14.3 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (1.16.4)
Requirement already satisfied: pandas<0.24,>=0.23.4 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.23.4)
Requirement already satisfied: parameterized<0.7.0,>=0.6.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (0.6.3)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (1.9.0)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from apache-beam==2.16.0.dev0) (5.0.4)
Requirement already satisfied: six>=1.5.2 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from grpcio<2,>=1.8->apache-beam==2.16.0.dev0) (1.12.0)
Requirement already satisfied: enum34>=1.0.4; python_version < "3.4" in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from grpcio<2,>=1.8->apache-beam==2.16.0.dev0) (1.1.6)
Requirement already satisfied: docopt in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (2.22.0)
Requirement already satisfied: pbr>=0.11 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.16.0.dev0) (5.4.2)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.16.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.16.0.dev0) (0.4.6)
Requirement already satisfied: pyasn1-modules>=0.0.5 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.16.0.dev0) (0.2.6)
Requirement already satisfied: rsa>=3.1.4 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam==2.16.0.dev0) (4.0)
Requirement already satisfied: setuptools in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from protobuf<4,>=3.5.0.post1->apache-beam==2.16.0.dev0) (41.0.1)
Requirement already satisfied: pyparsing>=2.1.4 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from pydot<2,>=1.2.0->apache-beam==2.16.0.dev0) (2.4.2)
Requirement already satisfied: monotonic>=0.6; python_version == "2.7" in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from tenacity<6.0,>=5.0.2->apache-beam==2.16.0.dev0) (1.5)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (1.25.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (2019.6.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155/lib/python2.7/site-packages> (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (2.8)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.16.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/45127155>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO:root:Writing 1000 documents to mongodb
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/mongodbio_it_test.py>:68: FutureWarning: WriteToMongoDB is experimental.
known_args.batch_size)
INFO:root:Writing 1000 documents to mongodb finished in 0.131 seconds
INFO:root:==================== <function annotate_downstream_side_inputs at 0x7fc9004df758> ====================
INFO:root:==================== <function fix_side_input_pcoll_coders at 0x7fc9004df848> ====================
INFO:root:==================== <function lift_combiners at 0x7fc9004df8c0> ====================
INFO:root:==================== <function expand_sdf at 0x7fc9004df938> ====================
INFO:root:==================== <function expand_gbk at 0x7fc9004df9b0> ====================
INFO:root:==================== <function sink_flattens at 0x7fc9004dfaa0> ====================
INFO:root:==================== <function greedily_fuse at 0x7fc9004dfb18> ====================
INFO:root:==================== <function read_to_impulse at 0x7fc9004dfb90> ====================
INFO:root:==================== <function impulse_to_input at 0x7fc9004dfc08> ====================
INFO:root:==================== <function inject_timer_pcollections at 0x7fc9004dfd70> ====================
INFO:root:==================== <function sort_stages at 0x7fc9004dfde8> ====================
INFO:root:==================== <function window_pcollection_coders at 0x7fc9004dfe60> ====================
INFO:root:Running (ref_AppliedPTransform_Create documents/Read_3)+((ref_AppliedPTransform_WriteToMongoDB/ParDo(_GenerateObjectIdFn)_5)+((ref_AppliedPTransform_WriteToMongoDB/Reshuffle/AddRandomKeys_7)+((ref_AppliedPTransform_WriteToMongoDB/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_9)+(WriteToMongoDB/Reshuffle/ReshufflePerKey/GroupByKey/Write))))
INFO:root:Running ((WriteToMongoDB/Reshuffle/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_WriteToMongoDB/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_14))+((ref_AppliedPTransform_WriteToMongoDB/Reshuffle/RemoveRandomKeys_15)+(ref_AppliedPTransform_WriteToMongoDB/ParDo(_WriteMongoFn)_16))
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO:root:Reading from mongodb beam_mongodbio_it_db:integration_test_1565287067
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/mongodbio_it_test.py>:82: FutureWarning: ReadFromMongoDB is experimental.
| 'Map' >> beam.Map(lambda doc: doc['number'])
INFO:root:Read 1000 documents from mongodb finished in 0.066 seconds
INFO:root:==================== <function annotate_downstream_side_inputs at 0x7fc9004df758> ====================
INFO:root:==================== <function fix_side_input_pcoll_coders at 0x7fc9004df848> ====================
INFO:root:==================== <function lift_combiners at 0x7fc9004df8c0> ====================
INFO:root:==================== <function expand_sdf at 0x7fc9004df938> ====================
INFO:root:==================== <function expand_gbk at 0x7fc9004df9b0> ====================
INFO:root:==================== <function sink_flattens at 0x7fc9004dfaa0> ====================
INFO:root:==================== <function greedily_fuse at 0x7fc9004dfb18> ====================
INFO:root:==================== <function read_to_impulse at 0x7fc9004dfb90> ====================
INFO:root:==================== <function impulse_to_input at 0x7fc9004dfc08> ====================
INFO:root:==================== <function inject_timer_pcollections at 0x7fc9004dfd70> ====================
INFO:root:==================== <function sort_stages at 0x7fc9004dfde8> ====================
INFO:root:==================== <function window_pcollection_coders at 0x7fc9004dfe60> ====================
INFO:root:Running ((ref_AppliedPTransform_assert_that/Create/Read_7)+((ref_AppliedPTransform_assert_that/Group/pair_with_0_11)+(assert_that/Group/Flatten/Transcode/0)))+(assert_that/Group/Flatten/Write/0)
INFO:root:Running ((ref_AppliedPTransform_ReadFromMongoDB/Read_3)+((ref_AppliedPTransform_Map_4)+((ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_8)+((ref_AppliedPTransform_assert_that/ToVoidKey_9)+((ref_AppliedPTransform_assert_that/Group/pair_with_1_12)+(assert_that/Group/Flatten/Transcode/1))))))+(assert_that/Group/Flatten/Write/1)
INFO:root:Running (assert_that/Group/Flatten/Read)+(assert_that/Group/GroupByKey/Write)
INFO:root:Running (assert_that/Group/GroupByKey/Read)+((ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_18)+((ref_AppliedPTransform_assert_that/Unkey_19)+(ref_AppliedPTransform_assert_that/Match_20)))
mongoioit27836
mongoioit27836
> Task :sdks:python:container:docker
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (1.7)
Requirement already satisfied: dill<0.2.10,>=0.2.9 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.2.9)
Requirement already satisfied: fastavro<0.22,>=0.21.4 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.21.4)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.16.0)
Requirement already satisfied: grpcio<2,>=1.8 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (1.15.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.12.0,>=0.8 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.9.2)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (2.0.0)
Collecting pymongo<4.0.0,>=3.8.0 (from apache-beam==2.16.0.dev0)
Downloading https://files.pythonhosted.org/packages/6e/ea/353a9f8ced71b9e258438aa4d6aa048c1eaf5b774d3ef2273d686fde0947/pymongo-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl (410kB)
Requirement already satisfied: oauth2client<4,>=2.0.1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (3.0.0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (3.6.1)
Requirement already satisfied: pydot<2,>=1.2.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (1.2.4)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (2.8.0)
Requirement already satisfied: pytz>=2018.3 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (3.12)
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (1.8.2)
Requirement already satisfied: futures<4.0.0,>=3.2.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (3.2.0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.6.8)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (3.6.1)
Requirement already satisfied: pyarrow<0.15.0,>=0.11.1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.11.1)
Requirement already satisfied: cachetools<4,>=3.1.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (3.1.1)
Collecting google-apitools<0.5.29,>=0.5.28 (from apache-beam==2.16.0.dev0)
Downloading https://files.pythonhosted.org/packages/07/5e/3e04cb66f5ced9267a854184bb09863d85d199646ea8480fee26b4313a00/google_apitools-0.5.28-py2-none-any.whl (134kB)
Collecting google-cloud-datastore<1.8.0,>=1.7.1 (from apache-beam==2.16.0.dev0)
Downloading https://files.pythonhosted.org/packages/d0/aa/29cbcf8cf7d08ce2d55b9dce858f7c632b434cb6451bed17cb4275804217/google_cloud_datastore-1.7.4-py2.py3-none-any.whl (82kB)
Requirement already satisfied: google-cloud-pubsub<0.40.0,>=0.39.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.39.0)
Requirement already satisfied: google-cloud-bigquery<1.18.0,>=1.6.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (1.6.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.28.1)
Requirement already satisfied: google-cloud-bigtable<0.33.0,>=0.31.1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.31.1)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore<7.1,>=7.0.1 in /usr/local/lib/python2.7/site-packages (from apache-beam==2.16.0.dev0) (7.0.1)
Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python2.7/site-packages (from grpcio<2,>=1.8->apache-beam==2.16.0.dev0) (1.12.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/site-packages (from grpcio<2,>=1.8->apache-beam==2.16.0.dev0) (1.1.6)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (2.22.0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (0.6.2)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /usr/local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.16.0.dev0) (1.0.2)
Requirement already satisfied: pbr>=0.11 in /usr/local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.16.0.dev0) (5.4.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/site-packages (from oauth2client<4,>=2.0.1->apache-beam==2.16.0.dev0) (0.4.6)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /usr/local/lib/python2.7/site-packages (from oauth2client<4,>=2.0.1->apache-beam==2.16.0.dev0) (0.2.6)
Requirement already satisfied: rsa>=3.1.4 in /usr/local/lib/python2.7/site-packages (from oauth2client<4,>=2.0.1->apache-beam==2.16.0.dev0) (4.0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.16.0.dev0) (39.1.0)
Requirement already satisfied: pyparsing>=2.1.4 in /usr/local/lib/python2.7/site-packages (from pydot<2,>=1.2.0->apache-beam==2.16.0.dev0) (2.3.1)
Requirement already satisfied: numpy>=1.14 in /usr/local/lib/python2.7/site-packages (from pyarrow<0.15.0,>=0.11.1->apache-beam==2.16.0.dev0) (1.15.2)
Requirement already satisfied: fasteners>=0.14 in /usr/local/lib/python2.7/site-packages (from google-apitools<0.5.29,>=0.5.28->apache-beam==2.16.0.dev0) (0.15)
Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.6.0 in /usr/local/lib/python2.7/site-packages (from google-cloud-datastore<1.8.0,>=1.7.1->apache-beam==2.16.0.dev0) (1.14.2)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.4 in /usr/local/lib/python2.7/site-packages (from google-cloud-pubsub<0.40.0,>=0.39.0->apache-beam==2.16.0.dev0) (0.11.4)
Requirement already satisfied: google-resumable-media>=0.2.1 in /usr/local/lib/python2.7/site-packages (from google-cloud-bigquery<1.18.0,>=1.6.0->apache-beam==2.16.0.dev0) (0.3.2)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /usr/local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.16.0.dev0) (1.6.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (1.25.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.16.0.dev0) (2019.6.16)
Requirement already satisfied: monotonic>=0.1 in /usr/local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<0.5.29,>=0.5.28->apache-beam==2.16.0.dev0) (1.5)
Requirement already satisfied: google-auth<2.0dev,>=0.4.0 in /usr/local/lib/python2.7/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-datastore<1.8.0,>=1.7.1->apache-beam==2.16.0.dev0) (1.6.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): still running...
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.16.0.dev0-cp27-cp27mu-linux_x86_64.whl size=5942627 sha256=8133beca9a66cd042c1b0513a098962692a9d710fa52db2af23e031edece6e4d
Stored in directory: /root/.cache/pip/wheels/4b/7d/dd/a19128d4b755625dce0a4a284246bffb9f82e49379d8e9d5f5
Successfully built apache-beam
[91mERROR: google-cloud-datastore 1.7.4 has requirement google-cloud-core<2.0dev,>=0.29.0, but you'll have google-cloud-core 0.28.1 which is incompatible.
[0mInstalling collected packages: pymongo, google-apitools, google-cloud-datastore, apache-beam
Found existing installation: google-apitools 0.5.26
Uninstalling google-apitools-0.5.26:
Successfully uninstalled google-apitools-0.5.26
Successfully installed apache-beam-2.16.0.dev0 google-apitools-0.5.28 google-cloud-datastore-1.7.4 pymongo-3.8.0
Removing intermediate container 74e65362ac57
---> d8e8c9ee70e3
Step 8/9 : ADD target/launcher/linux_amd64/boot /opt/apache/beam/
---> 2cd229c14765
Step 9/9 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in 3a278d9f1d87
Removing intermediate container 3a278d9f1d87
---> 1dd59c60625d
Successfully built 1dd59c60625d
Successfully tagged jenkins-docker-apache.bintray.io/beam/python:latest
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py2:setupVirtualenv'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py2:setupVirtualenv'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 56s
106 actionable tasks: 81 executed, 22 from cache, 3 up-to-date
Publishing build scan...
https://gradle.com/s/l4mb4a6sxck6a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org