You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/02/12 00:09:48 UTC
Build failed in Jenkins: beam_PostCommit_Python_PVR_Reference #3
See <https://builds.apache.org/job/beam_PostCommit_Python_PVR_Reference/3/display/redirect>
------------------------------------------
[...truncated 314.65 KB...]
[grpc-default-executor-1] INFO sdk_worker_main.main - Logging handler created.
[grpc-default-executor-1] INFO sdk_worker_main.start - Status HTTP server running at localhost:40157
[grpc-default-executor-1] INFO sdk_worker_main.main - semi_persistent_directory: /tmp
[grpc-default-executor-1] WARN sdk_worker_main._load_main_session - No session file found: /tmp/staged/pickled_main_session. Functions defined in __main__ (interactive session) may fail.
[grpc-default-executor-1] WARN pipeline_options.get_all_options - Discarding unparseable args: [u'--enforce_encodability', u'--direct_runner_use_stacked_bundle', u'--enforce_immutability', u'--pipeline_type_check', u'--block_on_run']
[grpc-default-executor-1] INFO sdk_worker_main.main - Python sdk harness started with pipeline_options: {'experiments': [u'beam_fn_api'], 'environment_type': u'DOCKER', 'sdk_location': u'container', 'job_name': u'test_sdf_1549930157.54', 'job_endpoint': u'localhost:43755'}
[grpc-default-executor-1] INFO sdk_worker.__init__ - Creating insecure control channel for localhost:32937.
[grpc-default-executor-1] INFO sdk_worker.__init__ - Control channel established.
[grpc-default-executor-1] INFO sdk_worker.__init__ - Initializing SDKHarness with 12 workers.
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.control.FnApiControlClientPoolService - Beam Fn Control client connected with id 1
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - Creating insecure state channel for .
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - State channel established.
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.data.GrpcDataService - Beam Fn Data client connected.
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_19:0:0
[grpc-default-executor-1] INFO data_plane.create_data_channel - Creating channel for localhost:34865
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_4:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_12:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_19:1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_7:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_22:0
[grpc-default-executor-1] ERROR sdk_worker._execute - Error processing instruction 16. Original traceback is
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 151, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 186, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 265, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 281, in process_bundle
delayed_applications = bundle_processor.process_bundle(instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 536, in process_bundle
].process_encoded(data.data)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 139, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 222, in apache_beam.runners.worker.operations.Operation.output
def output(self, windowed_value, output_index=0):
File "apache_beam/runners/worker/operations.py", line 223, in apache_beam.runners.worker.operations.Operation.output
cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 762, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 555, in apache_beam.runners.common.PerWindowInvoker.invoke_process
self._invoke_per_window(
File "apache_beam/runners/common.py", line 624, in apache_beam.runners.common.PerWindowInvoker._invoke_per_window
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 808, in apache_beam.runners.common._OutputProcessor.process_outputs
def process_outputs(self, windowed_input_element, results):
File "apache_beam/runners/common.py", line 847, in apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 762, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 418, in apache_beam.runners.common.SimpleInvoker.invoke_process
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 847, in apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 777, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise_with_traceback(new_exn)
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 419, in apache_beam.runners.common.SimpleInvoker.invoke_process
windowed_value, self.process_method(windowed_value.value))
File "apache_beam/transforms/core.py", line 1161, in <lambda>
wrapper = lambda x: [fn(x)]
File "apache_beam/testing/util.py", line 128, in _equal
'Failed assert: %r == %r' % (sorted_expected, sorted_actual))
BeamAssertException: Failed assert: ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z'] == ['a', 'b', 'd', 'e', 'p', 'q', 'w', 'x'] [while running 'assert_that/Match']
[reference-runner-pipeline-1] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[grpc-default-executor-1] INFO sdk_worker.run - No more requests from control plane
[grpc-default-executor-1] INFO sdk_worker.run - SDK Harness waiting for in-flight requests to complete
[grpc-default-executor-1] INFO data_plane.close - Closing all cached grpc data channels.
[grpc-default-executor-1] INFO sdk_worker.close - Closing all cached gRPC state handlers.
[grpc-default-executor-1] INFO sdk_worker.run - Done consuming work.
[grpc-default-executor-1] INFO sdk_worker_main.main - Python sdk harness exiting.
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Logging client hanged up.
EINFO:root:Using latest locally built Python SDK docker image.
INFO:root:==================== <function lift_combiners at 0x7f6395f4f320> ====================
[grpc-default-executor-2] ERROR org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService - Encountered Unexpected Exception for Invocation job-1204275486
java.lang.IllegalArgumentException: Can't get the number of an unknown enum value.
at org.apache.beam.model.jobmanagement.v1.JobApi$JobState$Enum.getNumber(JobApi.java:8585)
at org.apache.beam.model.jobmanagement.v1.JobApi$GetJobStateResponse$Builder.setState(JobApi.java:5768)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.getStateStream(ReferenceRunnerJobService.java:237)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:693)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Feb 12, 2019 12:09:33 AM org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@3c9b523d
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p13p1.com.google.common.base.Preconditions.checkState(Preconditions.java:444)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:172)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onCompleted(ServerCalls.java:358)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.getStateStream(ReferenceRunnerJobService.java:249)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:693)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[direct-runner-worker] WARN org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.environment.DockerCommand - Unable to pull docker image jenkins-docker-apache.bintray.io/beam/python:latest, cause: Received exit code 1 for command 'docker pull jenkins-docker-apache.bintray.io/beam/python:latest'. stderr: Error response from daemon: unknown: Repo 'apache' was not found
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job-1204275486/MANIFEST
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Loading manifest for retrieval token /tmp/beam-artifact-staging/job-1204275486/MANIFEST
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job-1204275486/MANIFEST has 0 artifact locations
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job-1204275486/MANIFEST -> 0 artifacts
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Beam Fn Logging client connected.
[grpc-default-executor-1] INFO sdk_worker_main.main - Logging handler created.
[grpc-default-executor-1] INFO sdk_worker_main.start - Status HTTP server running at localhost:33081
[grpc-default-executor-1] INFO sdk_worker_main.main - semi_persistent_directory: /tmp
[grpc-default-executor-1] WARN sdk_worker_main._load_main_session - No session file found: /tmp/staged/pickled_main_session. Functions defined in __main__ (interactive session) may fail.
[grpc-default-executor-1] WARN pipeline_options.get_all_options - Discarding unparseable args: [u'--enforce_encodability', u'--direct_runner_use_stacked_bundle', u'--enforce_immutability', u'--pipeline_type_check', u'--block_on_run']
[grpc-default-executor-1] INFO sdk_worker_main.main - Python sdk harness started with pipeline_options: {'experiments': [u'beam_fn_api'], 'environment_type': u'DOCKER', 'sdk_location': u'container', 'job_name': u'test_windowing_1549930171.94', 'job_endpoint': u'localhost:43755'}
[grpc-default-executor-1] INFO sdk_worker.__init__ - Creating insecure control channel for localhost:43025.
[grpc-default-executor-1] INFO sdk_worker.__init__ - Control channel established.
[grpc-default-executor-1] INFO sdk_worker.__init__ - Initializing SDKHarness with 12 workers.
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.control.FnApiControlClientPoolService - Beam Fn Control client connected with id 1
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - Creating insecure state channel for .
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - State channel established.
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_4:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_24:0:0
[grpc-default-executor-1] INFO data_plane.create_data_channel - Creating channel for localhost:34113
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.data.GrpcDataService - Beam Fn Data client connected.
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_17:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_12:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_7:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_24:1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_15:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_27:0
[grpc-default-executor-1] INFO sdk_worker.run - No more requests from control plane
[grpc-default-executor-1] INFO sdk_worker.run - SDK Harness waiting for in-flight requests to complete
[grpc-default-executor-3] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[grpc-default-executor-1] INFO data_plane.close - Closing all cached grpc data channels.
[grpc-default-executor-1] INFO sdk_worker.close - Closing all cached gRPC state handlers.
[grpc-default-executor-1] INFO sdk_worker.run - Done consuming work.
[grpc-default-executor-1] INFO sdk_worker_main.main - Python sdk harness exiting.
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Logging client hanged up.
[reference-runner-pipeline-1] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job-1204275486/MANIFEST has 0 artifact locations
[reference-runner-pipeline-1] INFO org.apache.beam.sdk.io.LocalFileSystem - Ignoring failed deletion of file /tmp/beam-artifact-staging/job-1204275486/artifacts/ which already does not exist: {}
java.nio.file.NoSuchFileException: /tmp/beam-artifact-staging/job-1204275486/artifacts
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at org.apache.beam.sdk.io.LocalFileSystem.delete(LocalFileSystem.java:185)
at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:379)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService.removeArtifacts(BeamFileSystemArtifactStagingService.java:169)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.lambda$run$3(ReferenceRunnerJobService.java:194)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService - Removed dir /tmp/beam-artifact-staging/job-1204275486/
.
======================================================================
ERROR: test_sdf (__main__.JavaReferenceRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 390, in test_sdf
assert_that(actual, equal_to(list(''.join(data))))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 412, in wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline job-1609834605 failed in state FAILED: unknown error
----------------------------------------------------------------------
Ran 29 tests in 200.271s
FAILED (errors=1, skipped=16)
> Task :beam-sdks-python:javaReferenceRunnerValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python_PVR_Reference/ws/src/sdks/python/build.gradle'> line: 380
* What went wrong:
Execution failed for task ':beam-sdks-python:javaReferenceRunnerValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 9m 26s
61 actionable tasks: 46 executed, 14 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/4hvtspiol5uro
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PostCommit_Python_PVR_Reference #6
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_PVR_Reference/6/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_PVR_Reference #5
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_PVR_Reference/5/display/redirect?page=changes>
Changes:
[samuelw] Improve visibility of large key commits in StreamingDataflowWorker by
[rohde.samuel] Modify ValidatesRunner to only run a custom test interface for user
[rohde.samuel] Fill in the ProcessBundleDescriptor with correct user timer information
[rohde.samuel] spotless
[rohde.samuel] fix timers to have correct kv coder
[rohde.samuel] Add timers to the ExecutionStage
[rohde.samuel] add initial Dataflow portable user timer implementation
[rohde.samuel] inform the SDK harness of the timers
[rohde.samuel] spotless
[rohde.samuel] clean up comments
[rohde.samuel] remove extra log lines
[rohde.samuel] remote try-catch in receive
[rohde.samuel] change todos to have jira
[rohde.samuel] Add user timer support in ParDoTest.java
[rohde.samuel] spotless
[rohde.samuel] fix merge
[rohde.samuel] remove dev files
[rohde.samuel] undo changes
[rohde.samuel] redundant logic
[rohde.samuel] undo changes?
[rohde.samuel] revert register node function (old unnecessary changes
[rohde.samuel] changes
[rohde.samuel] bad
[rohde.samuel] move fire timers to receive
[rohde.samuel] Update ParDoTest to add more supported timer tests to Dataflow
[rohde.samuel] spotless apply
[rohde.samuel] clean up
[rohde.samuel] finishing touches on user timers
[rohde.samuel] remove logger
[rohde.samuel] remove timer align bounded as supported
[rohde.samuel] remove force experiment
[rohde.samuel] experimental flag isn't in yet, remove supoprt
[rohde.samuel] Refactor timer implementation into its own class
[rohde.samuel] fix imports, and change string to byte encoder for timer test
[github] Use Dataflow dev container images that have current versions of Beam SDK
[robbe.sneyders] Unskip passing Python 3 tests
[robbe.sneyders] Skip failing typehinting tests for Python versions < 3.5.3
[kcweaver] [BEAM-3435] python examples use WriteToBigQuery instead of BigQuerySink
[markliu] Fix PostCommit Python 3 build failure
[robertwb] [BEAM-3837] More complete test for try-split.
[mxm] [BEAM-6418] Execute Flink tests serially on Jenkins to avoid memory
[markliu] Fix typo in Jenkins groovy name
[drieber] setMaxWorkItemCommitBytes must be only called for Streaming Engine, not
[drieber] Added test for streaming engine.
[drieber] Applied spotless.
[robbe.sneyders] Add py3 cython tox environment
[scott] [BEAM-6653] Implement Lullz logging for the Java SDK (#7818)
[kmj] Add missing dependency in legacy worker
[aaltay] [BEAM-6658] Add kms_key to BigQuery transforms, pass to Dataflow (#7824)
------------------------------------------
[...truncated 315.57 KB...]
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.control.FnApiControlClientPoolService - Beam Fn Control client connected with id 1
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - Creating insecure state channel for .
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - State channel established.
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_4:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_19:0:0
[grpc-default-executor-1] INFO data_plane.create_data_channel - Creating channel for localhost:41129
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.data.GrpcDataService - Beam Fn Data client connected.
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_12:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_19:1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_7:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_22:0
[grpc-default-executor-2] ERROR sdk_worker._execute - Error processing instruction 16. Original traceback is
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 151, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 186, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 265, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 281, in process_bundle
delayed_applications = bundle_processor.process_bundle(instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 547, in process_bundle
].process_encoded(data.data)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 139, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 222, in apache_beam.runners.worker.operations.Operation.output
def output(self, windowed_value, output_index=0):
File "apache_beam/runners/worker/operations.py", line 223, in apache_beam.runners.worker.operations.Operation.output
cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 762, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 555, in apache_beam.runners.common.PerWindowInvoker.invoke_process
self._invoke_per_window(
File "apache_beam/runners/common.py", line 624, in apache_beam.runners.common.PerWindowInvoker._invoke_per_window
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 808, in apache_beam.runners.common._OutputProcessor.process_outputs
def process_outputs(self, windowed_input_element, results):
File "apache_beam/runners/common.py", line 847, in apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 762, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 418, in apache_beam.runners.common.SimpleInvoker.invoke_process
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 847, in apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 777, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise_with_traceback(new_exn)
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 419, in apache_beam.runners.common.SimpleInvoker.invoke_process
windowed_value, self.process_method(windowed_value.value))
File "apache_beam/transforms/core.py", line 1161, in <lambda>
wrapper = lambda x: [fn(x)]
File "apache_beam/testing/util.py", line 128, in _equal
'Failed assert: %r == %r' % (sorted_expected, sorted_actual))
BeamAssertException: Failed assert: ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z'] == ['a', 'b', 'd', 'e', 'p', 'q', 'w', 'x'] [while running 'assert_that/Match']
[reference-runner-pipeline-1] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - 1 Beam Fn Logging clients still connected during shutdown.
[grpc-default-executor-2] ERROR data_plane._read_inputs - Failed to read inputs in the data plane
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/data_plane.py", line 271, in _read_inputs
for elements in elements_iterator:
File "/usr/local/lib/python2.7/site-packages/grpc/_channel.py", line 366, in next
return self._next()
File "/usr/local/lib/python2.7/site-packages/grpc/_channel.py", line 357, in _next
raise self
_Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1550022742.817327553","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1099,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
[grpc-default-executor-2] INFO sdk_worker.run - No more requests from control plane
[grpc-default-executor-2] INFO sdk_worker.run - SDK Harness waiting for in-flight requests to complete
EINFO:root:Using latest locally built Python SDK docker image.
INFO:root:==================== <function lift_combiners at 0x7f9f2008d398> ====================
[grpc-default-executor-3] ERROR org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService - Encountered Unexpected Exception for Invocation job1604106838
java.lang.IllegalArgumentException: Can't get the number of an unknown enum value.
at org.apache.beam.model.jobmanagement.v1.JobApi$JobState$Enum.getNumber(JobApi.java:8585)
at org.apache.beam.model.jobmanagement.v1.JobApi$GetJobStateResponse$Builder.setState(JobApi.java:5768)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.getStateStream(ReferenceRunnerJobService.java:237)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:693)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Feb 13, 2019 1:52:34 AM org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@5488ae8a
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p13p1.com.google.common.base.Preconditions.checkState(Preconditions.java:444)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:172)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onCompleted(ServerCalls.java:358)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.getStateStream(ReferenceRunnerJobService.java:249)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:693)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[direct-runner-worker] WARN org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.environment.DockerCommand - Unable to pull docker image jenkins-docker-apache.bintray.io/beam/python:latest, cause: Received exit code 1 for command 'docker pull jenkins-docker-apache.bintray.io/beam/python:latest'. stderr: Error response from daemon: unknown: Repo 'apache' was not found
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job1604106838/MANIFEST
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Loading manifest for retrieval token /tmp/beam-artifact-staging/job1604106838/MANIFEST
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job1604106838/MANIFEST has 0 artifact locations
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job1604106838/MANIFEST -> 0 artifacts
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Beam Fn Logging client connected.
[grpc-default-executor-2] INFO sdk_worker_main.main - Logging handler created.
[grpc-default-executor-2] INFO sdk_worker_main.start - Status HTTP server running at localhost:39513
[grpc-default-executor-2] INFO sdk_worker_main.main - semi_persistent_directory: /tmp
[grpc-default-executor-2] WARN sdk_worker_main._load_main_session - No session file found: /tmp/staged/pickled_main_session. Functions defined in __main__ (interactive session) may fail.
[grpc-default-executor-2] WARN pipeline_options.get_all_options - Discarding unparseable args: [u'--enforce_encodability', u'--direct_runner_use_stacked_bundle', u'--enforce_immutability', u'--pipeline_type_check', u'--block_on_run']
[grpc-default-executor-2] INFO sdk_worker_main.main - Python sdk harness started with pipeline_options: {'experiments': [u'beam_fn_api'], 'environment_type': u'DOCKER', 'sdk_location': u'container', 'job_name': u'test_windowing_1550022753.43', 'job_endpoint': u'localhost:46451'}
[grpc-default-executor-2] INFO sdk_worker.__init__ - Creating insecure control channel for localhost:37265.
[grpc-default-executor-2] INFO sdk_worker.__init__ - Control channel established.
[grpc-default-executor-2] INFO sdk_worker.__init__ - Initializing SDKHarness with 12 workers.
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.control.FnApiControlClientPoolService - Beam Fn Control client connected with id 1
[grpc-default-executor-2] INFO sdk_worker.create_state_handler - Creating insecure state channel for .
[grpc-default-executor-2] INFO sdk_worker.create_state_handler - State channel established.
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_24:1:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_4:0
[grpc-default-executor-2] INFO data_plane.create_data_channel - Creating channel for localhost:43965
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.data.GrpcDataService - Beam Fn Data client connected.
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_17:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_1:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_12:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_7:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_24:0:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_15:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_27:0
[grpc-default-executor-2] INFO sdk_worker.run - No more requests from control plane
[grpc-default-executor-2] INFO sdk_worker.run - SDK Harness waiting for in-flight requests to complete
[grpc-default-executor-2] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[grpc-default-executor-2] INFO data_plane.close - Closing all cached grpc data channels.
[grpc-default-executor-2] INFO sdk_worker.close - Closing all cached gRPC state handlers.
[grpc-default-executor-2] INFO sdk_worker.run - Done consuming work.
[grpc-default-executor-2] INFO sdk_worker_main.main - Python sdk harness exiting.
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Logging client hanged up.
[reference-runner-pipeline-1] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job1604106838/MANIFEST has 0 artifact locations
[reference-runner-pipeline-1] INFO org.apache.beam.sdk.io.LocalFileSystem - Ignoring failed deletion of file /tmp/beam-artifact-staging/job1604106838/artifacts/ which already does not exist: {}
java.nio.file.NoSuchFileException: /tmp/beam-artifact-staging/job1604106838/artifacts
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at org.apache.beam.sdk.io.LocalFileSystem.delete(LocalFileSystem.java:185)
at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:379)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService.removeArtifacts(BeamFileSystemArtifactStagingService.java:169)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.lambda$run$3(ReferenceRunnerJobService.java:194)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService - Removed dir /tmp/beam-artifact-staging/job1604106838/
.
======================================================================
ERROR: test_sdf (__main__.JavaReferenceRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 394, in test_sdf
assert_that(actual, equal_to(list(''.join(data))))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 412, in wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline job--908275015 failed in state FAILED: unknown error
----------------------------------------------------------------------
Ran 29 tests in 194.402s
FAILED (errors=1, skipped=16)
> Task :beam-sdks-python:javaReferenceRunnerValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python_PVR_Reference/ws/src/sdks/python/build.gradle'> line: 387
* What went wrong:
Execution failed for task ':beam-sdks-python:javaReferenceRunnerValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 9m 8s
61 actionable tasks: 46 executed, 14 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/f754iiu6ecb5q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_PVR_Reference #4
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_PVR_Reference/4/display/redirect?page=changes>
Changes:
[robert] [BEAM-3306] Hash keys when lifting
------------------------------------------
[...truncated 313.15 KB...]
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job253789799/MANIFEST
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Loading manifest for retrieval token /tmp/beam-artifact-staging/job253789799/MANIFEST
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job253789799/MANIFEST has 0 artifact locations
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job253789799/MANIFEST -> 0 artifacts
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Beam Fn Logging client connected.
[grpc-default-executor-2] INFO sdk_worker_main.main - Logging handler created.
[grpc-default-executor-2] INFO sdk_worker_main.start - Status HTTP server running at localhost:43479
[grpc-default-executor-2] INFO sdk_worker_main.main - semi_persistent_directory: /tmp
[grpc-default-executor-2] WARN sdk_worker_main._load_main_session - No session file found: /tmp/staged/pickled_main_session. Functions defined in __main__ (interactive session) may fail.
[grpc-default-executor-2] WARN pipeline_options.get_all_options - Discarding unparseable args: [u'--enforce_encodability', u'--direct_runner_use_stacked_bundle', u'--enforce_immutability', u'--pipeline_type_check', u'--block_on_run']
[grpc-default-executor-2] INFO sdk_worker_main.main - Python sdk harness started with pipeline_options: {'experiments': [u'beam_fn_api'], 'environment_type': u'DOCKER', 'sdk_location': u'container', 'job_name': u'test_sdf_1549933485.72', 'job_endpoint': u'localhost:60671'}
[grpc-default-executor-2] INFO sdk_worker.__init__ - Creating insecure control channel for localhost:38983.
[grpc-default-executor-2] INFO sdk_worker.__init__ - Control channel established.
[grpc-default-executor-2] INFO sdk_worker.__init__ - Initializing SDKHarness with 12 workers.
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.control.FnApiControlClientPoolService - Beam Fn Control client connected with id 1
[grpc-default-executor-2] INFO sdk_worker.create_state_handler - Creating insecure state channel for .
[grpc-default-executor-2] INFO sdk_worker.create_state_handler - State channel established.
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_4:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_19:0:0
[grpc-default-executor-2] INFO data_plane.create_data_channel - Creating channel for localhost:39395
[grpc-default-executor-2] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.data.GrpcDataService - Beam Fn Data client connected.
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_12:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_1:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_19:1:0
[grpc-default-executor-2] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_7:0
[grpc-default-executor-3] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_22:0
[grpc-default-executor-3] ERROR sdk_worker._execute - Error processing instruction 16. Original traceback is
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 151, in _execute
response = task()
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 186, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 265, in do_instruction
request.instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 281, in process_bundle
delayed_applications = bundle_processor.process_bundle(instruction_id)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 536, in process_bundle
].process_encoded(data.data)
File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 139, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 222, in apache_beam.runners.worker.operations.Operation.output
def output(self, windowed_value, output_index=0):
File "apache_beam/runners/worker/operations.py", line 223, in apache_beam.runners.worker.operations.Operation.output
cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 762, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 555, in apache_beam.runners.common.PerWindowInvoker.invoke_process
self._invoke_per_window(
File "apache_beam/runners/common.py", line 624, in apache_beam.runners.common.PerWindowInvoker._invoke_per_window
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 808, in apache_beam.runners.common._OutputProcessor.process_outputs
def process_outputs(self, windowed_input_element, results):
File "apache_beam/runners/common.py", line 847, in apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 762, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 418, in apache_beam.runners.common.SimpleInvoker.invoke_process
output_processor.process_outputs(
File "apache_beam/runners/common.py", line 847, in apache_beam.runners.common._OutputProcessor.process_outputs
self.main_receivers.receive(windowed_value)
File "apache_beam/runners/worker/operations.py", line 131, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
self.consumer.process(windowed_value)
File "apache_beam/runners/worker/operations.py", line 537, in apache_beam.runners.worker.operations.DoOperation.process
with self.scoped_process_state:
File "apache_beam/runners/worker/operations.py", line 538, in apache_beam.runners.worker.operations.DoOperation.process
delayed_application = self.dofn_receiver.receive(o)
File "apache_beam/runners/common.py", line 723, in apache_beam.runners.common.DoFnRunner.receive
self.process(windowed_value)
File "apache_beam/runners/common.py", line 729, in apache_beam.runners.common.DoFnRunner.process
self._reraise_augmented(exn)
File "apache_beam/runners/common.py", line 777, in apache_beam.runners.common.DoFnRunner._reraise_augmented
raise_with_traceback(new_exn)
File "apache_beam/runners/common.py", line 727, in apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/common.py", line 419, in apache_beam.runners.common.SimpleInvoker.invoke_process
windowed_value, self.process_method(windowed_value.value))
File "apache_beam/transforms/core.py", line 1161, in <lambda>
wrapper = lambda x: [fn(x)]
File "apache_beam/testing/util.py", line 128, in _equal
'Failed assert: %r == %r' % (sorted_expected, sorted_actual))
BeamAssertException: Failed assert: ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z'] == ['a', 'b', 'd', 'e', 'p', 'q', 'w', 'x'] [while running 'assert_that/Match']
[reference-runner-pipeline-1] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[grpc-default-executor-3] INFO sdk_worker.run - No more requests from control plane
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - 1 Beam Fn Logging clients still connected during shutdown.
EINFO:root:Using latest locally built Python SDK docker image.
INFO:root:==================== <function lift_combiners at 0x7f0ea184d320> ====================
[grpc-default-executor-1] ERROR org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService - Encountered Unexpected Exception for Invocation job-1108001263
java.lang.IllegalArgumentException: Can't get the number of an unknown enum value.
at org.apache.beam.model.jobmanagement.v1.JobApi$JobState$Enum.getNumber(JobApi.java:8585)
at org.apache.beam.model.jobmanagement.v1.JobApi$GetJobStateResponse$Builder.setState(JobApi.java:5768)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.getStateStream(ReferenceRunnerJobService.java:237)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:693)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Feb 12, 2019 1:05:01 AM org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@745a747e
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p13p1.com.google.common.base.Preconditions.checkState(Preconditions.java:444)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:172)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onCompleted(ServerCalls.java:358)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.getStateStream(ReferenceRunnerJobService.java:249)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:693)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[direct-runner-worker] WARN org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.environment.DockerCommand - Unable to pull docker image jenkins-docker-apache.bintray.io/beam/python:latest, cause: Received exit code 1 for command 'docker pull jenkins-docker-apache.bintray.io/beam/python:latest'. stderr: Error response from daemon: unknown: Repo 'apache' was not found
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job-1108001263/MANIFEST
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Loading manifest for retrieval token /tmp/beam-artifact-staging/job-1108001263/MANIFEST
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job-1108001263/MANIFEST has 0 artifact locations
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - GetManifest for /tmp/beam-artifact-staging/job-1108001263/MANIFEST -> 0 artifacts
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Beam Fn Logging client connected.
[grpc-default-executor-1] INFO sdk_worker_main.main - Logging handler created.
[grpc-default-executor-1] INFO sdk_worker_main.start - Status HTTP server running at localhost:45111
[grpc-default-executor-1] INFO sdk_worker_main.main - semi_persistent_directory: /tmp
[grpc-default-executor-1] WARN sdk_worker_main._load_main_session - No session file found: /tmp/staged/pickled_main_session. Functions defined in __main__ (interactive session) may fail.
[grpc-default-executor-1] WARN pipeline_options.get_all_options - Discarding unparseable args: [u'--enforce_encodability', u'--direct_runner_use_stacked_bundle', u'--enforce_immutability', u'--pipeline_type_check', u'--block_on_run']
[grpc-default-executor-1] INFO sdk_worker_main.main - Python sdk harness started with pipeline_options: {'experiments': [u'beam_fn_api'], 'environment_type': u'DOCKER', 'sdk_location': u'container', 'job_name': u'test_windowing_1549933500.06', 'job_endpoint': u'localhost:60671'}
[grpc-default-executor-1] INFO sdk_worker.__init__ - Creating insecure control channel for localhost:41137.
[grpc-default-executor-1] INFO sdk_worker.__init__ - Control channel established.
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.control.FnApiControlClientPoolService - Beam Fn Control client connected with id 1
[grpc-default-executor-1] INFO sdk_worker.__init__ - Initializing SDKHarness with 12 workers.
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - Creating insecure state channel for .
[grpc-default-executor-1] INFO sdk_worker.create_state_handler - State channel established.
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_4:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_24:0:0
[grpc-default-executor-1] INFO data_plane.create_data_channel - Creating channel for localhost:36843
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.data.GrpcDataService - Beam Fn Data client connected.
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_17:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_12:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_7:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_24:1:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_15:0
[grpc-default-executor-1] WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_27:0
[grpc-default-executor-1] INFO sdk_worker.run - No more requests from control plane
[grpc-default-executor-1] INFO sdk_worker.run - SDK Harness waiting for in-flight requests to complete
[grpc-default-executor-1] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[grpc-default-executor-1] INFO data_plane.close - Closing all cached grpc data channels.
[grpc-default-executor-1] INFO sdk_worker.close - Closing all cached gRPC state handlers.
[grpc-default-executor-1] INFO sdk_worker.run - Done consuming work.
[grpc-default-executor-1] INFO sdk_worker_main.main - Python sdk harness exiting.
[grpc-default-executor-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.logging.GrpcLoggingService - Logging client hanged up.
[reference-runner-pipeline-1] WARN org.apache.beam.repackaged.beam_runners_direct_java.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint.
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactRetrievalService - Manifest at /tmp/beam-artifact-staging/job-1108001263/MANIFEST has 0 artifact locations
[reference-runner-pipeline-1] INFO org.apache.beam.sdk.io.LocalFileSystem - Ignoring failed deletion of file /tmp/beam-artifact-staging/job-1108001263/artifacts/ which already does not exist: {}
java.nio.file.NoSuchFileException: /tmp/beam-artifact-staging/job-1108001263/artifacts
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at org.apache.beam.sdk.io.LocalFileSystem.delete(LocalFileSystem.java:185)
at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:379)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService.removeArtifacts(BeamFileSystemArtifactStagingService.java:169)
at org.apache.beam.runners.direct.portable.job.ReferenceRunnerJobService.lambda$run$3(ReferenceRunnerJobService.java:194)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[reference-runner-pipeline-1] INFO org.apache.beam.repackaged.beam_runners_direct_java.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService - Removed dir /tmp/beam-artifact-staging/job-1108001263/
.
======================================================================
ERROR: test_sdf (__main__.JavaReferenceRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 390, in test_sdf
assert_that(actual, equal_to(list(''.join(data))))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 412, in wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline job-835558094 failed in state FAILED: unknown error
----------------------------------------------------------------------
Ran 29 tests in 195.233s
FAILED (errors=1, skipped=16)
> Task :beam-sdks-python:javaReferenceRunnerValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python_PVR_Reference/ws/src/sdks/python/build.gradle'> line: 380
* What went wrong:
Execution failed for task ':beam-sdks-python:javaReferenceRunnerValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 8m 49s
61 actionable tasks: 46 executed, 14 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/xbbmsbcnuvqmw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org