You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/04/19 18:07:18 UTC
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4516
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4516/display/redirect?page=changes>
Changes:
[Elliotte Rusty Harold] use toMinutes
[Elliotte Rusty Harold] update test
[noreply] Change kafka table provider properties structure. (#14507)
------------------------------------------
[...truncated 647.42 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7fa836759400>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1618855633.7357237_64ef2e18-2788-43c5-b7c2-4ea68dc0b5ef failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_aa3c20ba-affd-43ee-b73c-7a7234592b15.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_aa3c20ba-affd-43ee-b73c-7a7234592b15.ref_Environment_default_environment_1.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_aa3c20ba-affd-43ee-b73c-7a7234592b15.null.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_aa3c20ba-affd-43ee-b73c-7a7234592b15.
21/04/19 18:07:14 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1618855633.7357237_64ef2e18-2788-43c5-b7c2-4ea68dc0b5ef
21/04/19 18:07:14 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1618855633.7357237_64ef2e18-2788-43c5-b7c2-4ea68dc0b5ef
21/04/19 18:07:14 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1618855633.7357237_64ef2e18-2788-43c5-b7c2-4ea68dc0b5ef.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7fa836486640>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1618855634.3258946_4430f286-26ab-4ffd-8c29-f8d6eb993fef failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_23916304-01f7-43e3-a09b-ab63464f9356.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_23916304-01f7-43e3-a09b-ab63464f9356.ref_Environment_default_environment_1.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_23916304-01f7-43e3-a09b-ab63464f9356.null.
21/04/19 18:07:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_23916304-01f7-43e3-a09b-ab63464f9356.
21/04/19 18:07:14 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1618855634.3258946_4430f286-26ab-4ffd-8c29-f8d6eb993fef
21/04/19 18:07:14 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1618855634.3258946_4430f286-26ab-4ffd-8c29-f8d6eb993fef
21/04/19 18:07:14 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1618855634.3258946_4430f286-26ab-4ffd-8c29-f8d6eb993fef.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 23.83 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 58s
70 actionable tasks: 46 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/gbhenwghx25ye
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python_VR_Spark
#4527
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4527/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4526
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4526/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12191] Add a test for python template generation with upload_graph
[noreply] [BEAM-7372] fix wrong usage of with_traceback (#14566)
[noreply] [BEAM-7372] cleanup codes for py2 from apache_beam/transforms (#14544)
[noreply] [BEAM-2085] Fixups for Python resource hints. (#14605)
------------------------------------------
[...truncated 629.61 KB...]
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1619071560.3219335_3bd77c8d-0d22-46f0-a199-9a622ad01024 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/22 06:06:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_eb7067e9-0065-4b12-a8e5-3aafd04540c7.
21/04/22 06:06:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_eb7067e9-0065-4b12-a8e5-3aafd04540c7.ref_Environment_default_environment_1.
21/04/22 06:06:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_eb7067e9-0065-4b12-a8e5-3aafd04540c7.null.
21/04/22 06:06:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_eb7067e9-0065-4b12-a8e5-3aafd04540c7.
21/04/22 06:06:00 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1619071560.3219335_3bd77c8d-0d22-46f0-a199-9a622ad01024
21/04/22 06:06:00 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1619071560.3219335_3bd77c8d-0d22-46f0-a199-9a622ad01024
21/04/22 06:06:00 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1619071560.3219335_3bd77c8d-0d22-46f0-a199-9a622ad01024.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:371 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7fb27d73fe80>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1619071561.0773165_fd511d04-d34e-414f-a6cd-13743db9773e failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/22 06:06:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_f652fd56-9b2d-4009-9a2e-e26a8e69b9e6.
21/04/22 06:06:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_f652fd56-9b2d-4009-9a2e-e26a8e69b9e6.ref_Environment_default_environment_1.
21/04/22 06:06:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_f652fd56-9b2d-4009-9a2e-e26a8e69b9e6.null.
21/04/22 06:06:01 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_f652fd56-9b2d-4009-9a2e-e26a8e69b9e6.
21/04/22 06:06:01 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1619071561.0773165_fd511d04-d34e-414f-a6cd-13743db9773e
21/04/22 06:06:01 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1619071561.0773165_fd511d04-d34e-414f-a6cd-13743db9773e
21/04/22 06:06:01 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1619071561.0773165_fd511d04-d34e-414f-a6cd-13743db9773e.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21/04/22 06:06:01 ERROR org.apache.beam.runners.jobsubmission.InMemoryJobService: Encountered Unexpected Exception for Invocation test_windowing_1619071561.0773165_fd511d04-d34e-414f-a6cd-13743db9773e
java.lang.IllegalStateException: Stream is already completed, no further calls are allowed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onNext(ServerCalls.java:346)
at org.apache.beam.sdk.fn.stream.SynchronizedStreamObserver.onNext(SynchronizedStreamObserver.java:46)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$getMessageStream$2(InMemoryJobService.java:434)
at org.apache.beam.runners.jobsubmission.JobInvocation.addStateListener(JobInvocation.java:218)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:450)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
Apr 22, 2021 6:06:01 AM org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@69e6805e
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:209)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:202)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onError(ServerCalls.java:360)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:458)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:371 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 27.19 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 32s
70 actionable tasks: 46 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/bxsnnz5qlxasy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4525
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4525/display/redirect?page=changes>
Changes:
[sychen] Add a BQ option for configuring buffering duration when auto-sharding is
[suztomo] [BEAM-11010] Upgrading google-cloud-pubsublite to 0.13.2
[suztomo] [BEAM-11010] Copying SubscriberOptions from pubsublite repo
[suztomo] [BEAM-11010] Declaring flogger-system-backend to avoid conflicts
[Kyle Weaver] [BEAM-12194] Enable SqlTransform::registerUdaf in ZetaSQL.
[Kyle Weaver] [BEAM-12194] Code style changes from review.
[Boyuan Zhang] [BEAM-12114] Dataflow should apply KAFKA_READ_OVERRIDE when it's not
[noreply] Make sdk/worker_harness_container_image fully backwards compatible
------------------------------------------
[...truncated 705.46 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21/04/22 00:22:16 ERROR org.apache.beam.runners.jobsubmission.InMemoryJobService: Encountered Unexpected Exception for Invocation test_windowed_pardo_state_timers_1619050936.0436006_93149d07-fdcf-4914-8479-b5bb2958d8b6
java.lang.IllegalStateException: Stream is already completed, no further calls are allowed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onNext(ServerCalls.java:346)
at org.apache.beam.sdk.fn.stream.SynchronizedStreamObserver.onNext(SynchronizedStreamObserver.java:46)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$getMessageStream$2(InMemoryJobService.java:434)
at org.apache.beam.runners.jobsubmission.JobInvocation.addStateListener(JobInvocation.java:218)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:450)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 22, 2021 12:22:16 AM org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@7da5275f
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:209)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:202)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onError(ServerCalls.java:360)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:458)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f3275d7a2b0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1619050937.0741768_b157c73d-5ef7-418c-b945-5f6a3f11043f failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/22 00:22:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_c2315ef7-555a-4cad-a6eb-97607599743a.
21/04/22 00:22:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_c2315ef7-555a-4cad-a6eb-97607599743a.ref_Environment_default_environment_1.
21/04/22 00:22:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_c2315ef7-555a-4cad-a6eb-97607599743a.null.
21/04/22 00:22:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_c2315ef7-555a-4cad-a6eb-97607599743a.
21/04/22 00:22:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1619050937.0741768_b157c73d-5ef7-418c-b945-5f6a3f11043f
21/04/22 00:22:17 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1619050937.0741768_b157c73d-5ef7-418c-b945-5f6a3f11043f
21/04/22 00:22:17 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1619050937.0741768_b157c73d-5ef7-418c-b945-5f6a3f11043f.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21/04/22 00:22:17 ERROR org.apache.beam.runners.jobsubmission.InMemoryJobService: Encountered Unexpected Exception for Invocation test_windowing_1619050937.0741768_b157c73d-5ef7-418c-b945-5f6a3f11043f
java.lang.IllegalStateException: Stream is already completed, no further calls are allowed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onNext(ServerCalls.java:346)
at org.apache.beam.sdk.fn.stream.SynchronizedStreamObserver.onNext(SynchronizedStreamObserver.java:46)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$getMessageStream$2(InMemoryJobService.java:434)
at org.apache.beam.runners.jobsubmission.JobInvocation.addStateListener(JobInvocation.java:218)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:450)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 22, 2021 12:22:17 AM org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@1921c830
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:209)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:202)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onError(ServerCalls.java:360)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:458)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 47.61 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 21m 51s
70 actionable tasks: 62 executed, 8 from cache
Publishing build scan...
https://gradle.com/s/7qoc3vaj4z6js
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4524
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4524/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12009] Copy CalcRelSplitter.
[Kyle Weaver] [BEAM-12009] Implement Calc splitting rule.
[Kyle Weaver] [BEAM-12009] Use different method signature to avoid spurious null check
[Kyle Weaver] [BEAM-12009] Move CalcRelSplitter to rel subpackage.
[Kyle Weaver] [BEAM-12009] Reduce code duplication between rules by re-implementing
[Kyle Weaver] [BEAM-12009] Only match in BeamCalcSplittingRule if ≥ 1 of its component
[anup.d] BEAM-12166:Beam Sql - Combine Accumulator return Map fails with class
[suztomo] [BEAM-8357] Upgrading auto-value to 1.8 from 1.7.4
[Kenneth Knowles] Build source release zip from RC tag
[aromanenko.dev] [BEAM-12197] TPC-DS: Fix SQL-queries syntax
------------------------------------------
[...truncated 598.50 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f2f4a546c40>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1619028490.172412_8d24cdbd-7f0a-413d-b714-f5beed58ec27 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1cdc97f8-5d7e-4283-a628-cf314359df75.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1cdc97f8-5d7e-4283-a628-cf314359df75.ref_Environment_default_environment_1.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_1cdc97f8-5d7e-4283-a628-cf314359df75.null.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1cdc97f8-5d7e-4283-a628-cf314359df75.
21/04/21 18:08:10 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1619028490.172412_8d24cdbd-7f0a-413d-b714-f5beed58ec27
21/04/21 18:08:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1619028490.172412_8d24cdbd-7f0a-413d-b714-f5beed58ec27
21/04/21 18:08:10 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1619028490.172412_8d24cdbd-7f0a-413d-b714-f5beed58ec27.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f2f40a446a0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1619028490.6663888_825eda2c-a9ef-4633-b9e4-036cff9e6835 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_ad8aafd7-4ba6-45d9-bf34-da8399006950.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_ad8aafd7-4ba6-45d9-bf34-da8399006950.ref_Environment_default_environment_1.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_ad8aafd7-4ba6-45d9-bf34-da8399006950.null.
21/04/21 18:08:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_ad8aafd7-4ba6-45d9-bf34-da8399006950.
21/04/21 18:08:10 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1619028490.6663888_825eda2c-a9ef-4633-b9e4-036cff9e6835
21/04/21 18:08:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1619028490.6663888_825eda2c-a9ef-4633-b9e4-036cff9e6835
21/04/21 18:08:10 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1619028490.6663888_825eda2c-a9ef-4633-b9e4-036cff9e6835.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 22.23 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 57s
70 actionable tasks: 48 executed, 22 from cache
Publishing build scan...
https://gradle.com/s/ppwxuh3orszq6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4523
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4523/display/redirect>
Changes:
------------------------------------------
[...truncated 599.52 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7fa7ddb9d940>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1619006781.7238817_ace9cf39-189d-4a59-9db8-ef6baeff320d failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_bd3be8f9-7288-47a4-962b-d9075c2062f1.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_bd3be8f9-7288-47a4-962b-d9075c2062f1.ref_Environment_default_environment_1.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_bd3be8f9-7288-47a4-962b-d9075c2062f1.null.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_bd3be8f9-7288-47a4-962b-d9075c2062f1.
21/04/21 12:06:22 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1619006781.7238817_ace9cf39-189d-4a59-9db8-ef6baeff320d
21/04/21 12:06:22 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1619006781.7238817_ace9cf39-189d-4a59-9db8-ef6baeff320d
21/04/21 12:06:22 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1619006781.7238817_ace9cf39-189d-4a59-9db8-ef6baeff320d.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7fa7ddbaccd0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1619006782.2472734_c3f94321-d75b-47d4-a52d-0ed556cccd54 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_955e8db1-e759-4bef-89f3-6de7ee8cbf8a.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_955e8db1-e759-4bef-89f3-6de7ee8cbf8a.ref_Environment_default_environment_1.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_955e8db1-e759-4bef-89f3-6de7ee8cbf8a.null.
21/04/21 12:06:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_955e8db1-e759-4bef-89f3-6de7ee8cbf8a.
21/04/21 12:06:22 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1619006782.2472734_c3f94321-d75b-47d4-a52d-0ed556cccd54
21/04/21 12:06:22 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1619006782.2472734_c3f94321-d75b-47d4-a52d-0ed556cccd54
21/04/21 12:06:22 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1619006782.2472734_c3f94321-d75b-47d4-a52d-0ed556cccd54.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 20.47 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 52s
70 actionable tasks: 47 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/mmqxenwnavcqy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4522
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4522/display/redirect?page=changes>
Changes:
[noreply] Update stringSlice to impl pkg.go.dev/flag#Getter.Get
[noreply] [BEAM-12174] Samza Portable Runner Support (#14554)
------------------------------------------
[...truncated 603.12 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f69342bb5b0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1618985228.5206807_7e813427-5799-41c4-8a3c-0cb036e68164 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 06:07:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_5c0147de-3350-4b14-ad2d-68a44ab7508c.
21/04/21 06:07:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_5c0147de-3350-4b14-ad2d-68a44ab7508c.ref_Environment_default_environment_1.
21/04/21 06:07:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_5c0147de-3350-4b14-ad2d-68a44ab7508c.null.
21/04/21 06:07:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_5c0147de-3350-4b14-ad2d-68a44ab7508c.
21/04/21 06:07:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1618985228.5206807_7e813427-5799-41c4-8a3c-0cb036e68164
21/04/21 06:07:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1618985228.5206807_7e813427-5799-41c4-8a3c-0cb036e68164
21/04/21 06:07:08 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1618985228.5206807_7e813427-5799-41c4-8a3c-0cb036e68164.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f69342bd880>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1618985229.0293565_63b433c8-d677-4260-93e3-9ca94c680565 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 06:07:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_b5ff81f0-77ad-4f7b-af19-fe9ee65d1256.
21/04/21 06:07:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_b5ff81f0-77ad-4f7b-af19-fe9ee65d1256.ref_Environment_default_environment_1.
21/04/21 06:07:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_b5ff81f0-77ad-4f7b-af19-fe9ee65d1256.null.
21/04/21 06:07:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_b5ff81f0-77ad-4f7b-af19-fe9ee65d1256.
21/04/21 06:07:09 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1618985229.0293565_63b433c8-d677-4260-93e3-9ca94c680565
21/04/21 06:07:09 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1618985229.0293565_63b433c8-d677-4260-93e3-9ca94c680565
21/04/21 06:07:09 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1618985229.0293565_63b433c8-d677-4260-93e3-9ca94c680565.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 23.95 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 2s
70 actionable tasks: 47 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/s673qxblcwhz4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4521
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4521/display/redirect?page=changes>
Changes:
[kileysok] Update Java beam container image
[suztomo] Libraries-BOM 20.0.0 with gRPC and protobuf
[suztomo] Updating assertions for google-http-client 1.39.2
[suztomo] Declare gax-httpjson as a direct dependency of google-cloud-platform
[noreply] Add --sdk_container_image Python WorkerOption (#14557)
[noreply] [BEAM-2085] Add capability to declare resource hints in Python pipeline
------------------------------------------
[...truncated 662.03 KB...]
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f3e6d199220>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1618963817.1791742_6b5e84d7-8ea5-44f8-a427-a8c0dd07e6b9 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 00:10:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_21403ccc-b80d-427c-a0de-73542c1abe61.
21/04/21 00:10:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_21403ccc-b80d-427c-a0de-73542c1abe61.ref_Environment_default_environment_1.
21/04/21 00:10:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_21403ccc-b80d-427c-a0de-73542c1abe61.null.
21/04/21 00:10:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_21403ccc-b80d-427c-a0de-73542c1abe61.
21/04/21 00:10:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1618963817.1791742_6b5e84d7-8ea5-44f8-a427-a8c0dd07e6b9
21/04/21 00:10:17 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1618963817.1791742_6b5e84d7-8ea5-44f8-a427-a8c0dd07e6b9
21/04/21 00:10:17 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1618963817.1791742_6b5e84d7-8ea5-44f8-a427-a8c0dd07e6b9.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:583: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f3e6d066bb0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1618963817.7972572_63a45887-7916-468f-9155-4ec5a605726f failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/21 00:10:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_bfe1af8c-7714-4cb5-b1cf-7244e9ba9573.
21/04/21 00:10:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_bfe1af8c-7714-4cb5-b1cf-7244e9ba9573.ref_Environment_default_environment_1.
21/04/21 00:10:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_bfe1af8c-7714-4cb5-b1cf-7244e9ba9573.null.
21/04/21 00:10:18 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_bfe1af8c-7714-4cb5-b1cf-7244e9ba9573.
21/04/21 00:10:18 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1618963817.7972572_63a45887-7916-468f-9155-4ec5a605726f
21/04/21 00:10:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1618963817.7972572_63a45887-7916-468f-9155-4ec5a605726f
21/04/21 00:10:18 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1618963817.7972572_63a45887-7916-468f-9155-4ec5a605726f.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:373 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 23.26 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 46s
70 actionable tasks: 48 executed, 22 from cache
Gradle was unable to watch the file system for changes. The inotify watches limit is too low.
Publishing build scan...
https://gradle.com/s/fnbnfsnbxumuu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4520
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4520/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Populate runtime parameters in SDK worker startup.
[Robert Bradshaw] [BEAM-6597] Use runner capabilities to activate short ids.
[Robert Bradshaw] Fix wording.
[Robert Bradshaw] Comment about runtime value usage.
[noreply] Add tensorboard-data-server license URL for Python container image.
------------------------------------------
[...truncated 609.08 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f4f2e1faee0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1618943432.363611_9a0d4920-2639-4486-bb9b-2f68024389b9 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/20 18:30:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_f26e09eb-9da0-4ec3-a56e-1455d00b8e15.
21/04/20 18:30:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_f26e09eb-9da0-4ec3-a56e-1455d00b8e15.ref_Environment_default_environment_1.
21/04/20 18:30:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_f26e09eb-9da0-4ec3-a56e-1455d00b8e15.null.
21/04/20 18:30:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_f26e09eb-9da0-4ec3-a56e-1455d00b8e15.
21/04/20 18:30:32 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1618943432.363611_9a0d4920-2639-4486-bb9b-2f68024389b9
21/04/20 18:30:32 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1618943432.363611_9a0d4920-2639-4486-bb9b-2f68024389b9
21/04/20 18:30:32 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1618943432.363611_9a0d4920-2639-4486-bb9b-2f68024389b9.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f4f2e1ccdc0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1618943432.9540887_a35c543d-6c9a-454d-8323-c7ffc3c12d9b failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/20 18:30:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_c6317abf-66cc-432c-9bb9-e7568abbf011.
21/04/20 18:30:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_c6317abf-66cc-432c-9bb9-e7568abbf011.ref_Environment_default_environment_1.
21/04/20 18:30:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_c6317abf-66cc-432c-9bb9-e7568abbf011.null.
21/04/20 18:30:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_c6317abf-66cc-432c-9bb9-e7568abbf011.
21/04/20 18:30:33 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1618943432.9540887_a35c543d-6c9a-454d-8323-c7ffc3c12d9b
21/04/20 18:30:33 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1618943432.9540887_a35c543d-6c9a-454d-8323-c7ffc3c12d9b
21/04/20 18:30:33 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1618943432.9540887_a35c543d-6c9a-454d-8323-c7ffc3c12d9b.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 21.51 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 8s
70 actionable tasks: 47 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/n2fpmuqvpdaji
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4519
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4519/display/redirect>
Changes:
------------------------------------------
[...truncated 646.35 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f8e43e02d90>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1618920343.2955563_a5c8fb32-99f1-4208-b727-ca8d31de2b10 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/20 12:05:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d6b595ef-2a9c-46b5-93f6-58cbe4de0f93.
21/04/20 12:05:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d6b595ef-2a9c-46b5-93f6-58cbe4de0f93.ref_Environment_default_environment_1.
21/04/20 12:05:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_d6b595ef-2a9c-46b5-93f6-58cbe4de0f93.null.
21/04/20 12:05:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d6b595ef-2a9c-46b5-93f6-58cbe4de0f93.
21/04/20 12:05:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1618920343.2955563_a5c8fb32-99f1-4208-b727-ca8d31de2b10
21/04/20 12:05:43 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1618920343.2955563_a5c8fb32-99f1-4208-b727-ca8d31de2b10
21/04/20 12:05:43 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1618920343.2955563_a5c8fb32-99f1-4208-b727-ca8d31de2b10.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f8e43a517c0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1618920343.785441_8bb95476-dc90-4d5a-90cc-de7b6cf7b327 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/20 12:05:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9077e049-0a1a-4ee1-a16c-09b0fee8826b.
21/04/20 12:05:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9077e049-0a1a-4ee1-a16c-09b0fee8826b.ref_Environment_default_environment_1.
21/04/20 12:05:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_9077e049-0a1a-4ee1-a16c-09b0fee8826b.null.
21/04/20 12:05:44 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9077e049-0a1a-4ee1-a16c-09b0fee8826b.
21/04/20 12:05:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1618920343.785441_8bb95476-dc90-4d5a-90cc-de7b6cf7b327
21/04/20 12:05:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1618920343.785441_8bb95476-dc90-4d5a-90cc-de7b6cf7b327
21/04/20 12:05:44 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1618920343.785441_8bb95476-dc90-4d5a-90cc-de7b6cf7b327.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 27.74 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 16s
70 actionable tasks: 46 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/fktiemxuyx7ae
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4518
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4518/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12191] Fix a bug that upload_graph doesn't reduce template file
[noreply] [BEAM-10029] Spanner IO read and write performance tests (#13857)
------------------------------------------
[...truncated 723.01 KB...]
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21/04/20 06:06:33 ERROR org.apache.beam.runners.jobsubmission.InMemoryJobService: Encountered Unexpected Exception for Invocation test_windowed_pardo_state_timers_1618898793.2728379_0ca2772f-cb49-46d9-b64c-fe57df235f98
java.lang.IllegalStateException: Stream is already completed, no further calls are allowed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onNext(ServerCalls.java:346)
at org.apache.beam.sdk.fn.stream.SynchronizedStreamObserver.onNext(SynchronizedStreamObserver.java:46)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$getMessageStream$2(InMemoryJobService.java:434)
at org.apache.beam.runners.jobsubmission.JobInvocation.addStateListener(JobInvocation.java:218)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:450)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 20, 2021 6:06:33 AM org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@505ab41f
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:209)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:202)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onError(ServerCalls.java:360)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:458)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f6c9861b0d0>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1618898794.0935087_1f9068ef-e9eb-4513-8fe0-7095808fd494 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/20 06:06:34 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_eea7d0cd-a961-4f1a-88ff-be85d25643ec.
21/04/20 06:06:34 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_eea7d0cd-a961-4f1a-88ff-be85d25643ec.ref_Environment_default_environment_1.
21/04/20 06:06:34 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_eea7d0cd-a961-4f1a-88ff-be85d25643ec.null.
21/04/20 06:06:34 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_eea7d0cd-a961-4f1a-88ff-be85d25643ec.
21/04/20 06:06:34 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1618898794.0935087_1f9068ef-e9eb-4513-8fe0-7095808fd494
21/04/20 06:06:34 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1618898794.0935087_1f9068ef-e9eb-4513-8fe0-7095808fd494
21/04/20 06:06:34 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1618898794.0935087_1f9068ef-e9eb-4513-8fe0-7095808fd494.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21/04/20 06:06:34 ERROR org.apache.beam.runners.jobsubmission.InMemoryJobService: Encountered Unexpected Exception for Invocation test_windowing_1618898794.0935087_1f9068ef-e9eb-4513-8fe0-7095808fd494
java.lang.IllegalStateException: Stream is already completed, no further calls are allowed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onNext(ServerCalls.java:346)
at org.apache.beam.sdk.fn.stream.SynchronizedStreamObserver.onNext(SynchronizedStreamObserver.java:46)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$getMessageStream$2(InMemoryJobService.java:434)
at org.apache.beam.runners.jobsubmission.JobInvocation.addStateListener(JobInvocation.java:218)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:450)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 20, 2021 6:06:34 AM org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor run
SEVERE: Exception while executing runnable org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@8d9fa8
java.lang.IllegalStateException: call already closed
at org.apache.beam.vendor.grpc.v1p26p0.com.google.common.base.Preconditions.checkState(Preconditions.java:511)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:209)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:202)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onError(ServerCalls.java:360)
at org.apache.beam.runners.jobsubmission.InMemoryJobService.getMessageStream(InMemoryJobService.java:458)
at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:985)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 37.36 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 3s
70 actionable tasks: 46 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/cpiz3kq7pqaqq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_VR_Spark #4517
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4517/display/redirect?page=changes>
Changes:
[samuelw] [BEAM-12118] Fix racy precondition exception in QueueingBeamFnDataClient
[noreply] [BEAM-11839] Improve DeferredFrameTest._run_test (#14552)
------------------------------------------
[...truncated 606.91 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
_______________ SparkRunnerTest.test_windowed_pardo_state_timers _______________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowed_pardo_state_timers>
def test_windowed_pardo_state_timers(self):
> self._run_pardo_state_timers(windowed=True)
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:425:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:490: in _run_pardo_state_timers
assert_that(actual, is_buffered_correctly)
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f3070399d90>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowed_pardo_state_timers_1618877320.9043615_44bf95f5-2dcc-4ec2-ac83-20c1fc8c9cad failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_205b820b-ebb9-4e12-be57-56febd33dc4d.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_205b820b-ebb9-4e12-be57-56febd33dc4d.ref_Environment_default_environment_1.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_205b820b-ebb9-4e12-be57-56febd33dc4d.null.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_205b820b-ebb9-4e12-be57-56febd33dc4d.
21/04/20 00:08:41 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowed_pardo_state_timers_1618877320.9043615_44bf95f5-2dcc-4ec2-ac83-20c1fc8c9cad
21/04/20 00:08:41 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowed_pardo_state_timers_1618877320.9043615_44bf95f5-2dcc-4ec2-ac83-20c1fc8c9cad
21/04/20 00:08:41 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowed_pardo_state_timers_1618877320.9043615_44bf95f5-2dcc-4ec2-ac83-20c1fc8c9cad.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
________________________ SparkRunnerTest.test_windowing ________________________
self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_windowing>
def test_windowing(self):
with self.create_pipeline() as p:
res = (
p
| beam.Create([1, 2, 100, 101, 102])
| beam.Map(lambda t: window.TimestampedValue(('k', t), t))
| beam.WindowInto(beam.transforms.window.Sessions(10))
| beam.GroupByKey()
| beam.Map(lambda k_vs1: (k_vs1[0], sorted(k_vs1[1]))))
> assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:777:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/pipeline.py:580: in __exit__
self.result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7f306a999e50>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline test_windowing_1618877321.4319391_39e9ce19-e293-45f0-96b5-406fbc280cb0 failed in state FAILED: java.lang.NullPointerException
apache_beam/runners/portability/portable_runner.py:602: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_df10dfe5-0017-40fb-a862-16b74e6e4005.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_df10dfe5-0017-40fb-a862-16b74e6e4005.ref_Environment_default_environment_1.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 0 artifacts for job_df10dfe5-0017-40fb-a862-16b74e6e4005.null.
21/04/20 00:08:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_df10dfe5-0017-40fb-a862-16b74e6e4005.
21/04/20 00:08:41 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job test_windowing_1618877321.4319391_39e9ce19-e293-45f0-96b5-406fbc280cb0
21/04/20 00:08:41 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation test_windowing_1618877321.4319391_39e9ce19-e293-45f0-96b5-406fbc280cb0
21/04/20 00:08:41 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: Error during job invocation test_windowing_1618877321.4319391_39e9ce19-e293-45f0-96b5-406fbc280cb0.
java.lang.NullPointerException
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)
at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ERROR:root:java.lang.NullPointerException
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:316 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
ERROR root:portable_runner.py:569 java.lang.NullPointerException
=============================== warnings summary ===============================
target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> -
========= 33 failed, 1 passed, 17 skipped, 1 warnings in 22.42 seconds =========
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1)
[0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[31mERROR: spark-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 10s
70 actionable tasks: 47 executed, 23 from cache
Publishing build scan...
https://gradle.com/s/enro4u65sewzk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org