You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/10/06 18:37:48 UTC
Build failed in Jenkins: beam_PreCommit_Python_Cron #4765
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4765/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Restore "Default to Runner v2 for Python Streaming jobs. (#15140)"
[Robert Bradshaw] Avoid incompatible setting.
[noreply] [BEAM-12909][BEAM-12849] Add support for running spark3 nexmark queries
------------------------------------------
[...truncated 615.91 KB...]
Using cached pytz-2021.3-py2.py3-none-any.whl (503 kB)
Collecting requests<3.0.0,>=2.24.0
Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting typing-extensions<4,>=3.7.0
Using cached typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Collecting orjson<4.0
Using cached orjson-3.6.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (250 kB)
Collecting pandas<1.4,>=1.0
Using cached pandas-1.3.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.5 MB)
Collecting freezegun>=0.3.12
Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<6.0.0,>=3.12
Using cached PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.25-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.5.2 in ./target/.tox-py38-cython/py38-cython/lib/python3.8/site-packages (from grpcio<2,>=1.29.0->apache-beam==2.34.0.dev0) (1.16.0)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.6.0-py2.py3-none-any.whl (111 kB)
Collecting rsa>=3.1.4
Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pyasn1-modules>=0.0.5
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.10.0-py3-none-any.whl (51 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting py>=1.5.0
Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting packaging
Using cached packaging-21.0-py3-none-any.whl (40 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.3.0-py2.py3-none-any.whl (4.7 kB)
Collecting idna<4,>=2.5
Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (156 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.1-cp38-cp38-manylinux2010_x86_64.whl (84 kB)
Collecting docker
Using cached docker-5.0.2-py2.py3-none-any.whl (145 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.1-py2.py3-none-any.whl (52 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
=================================== log end ====================================
___________________________________ summary ____________________________________
[31mERROR: py38-cython: InvocationError for command <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-cython/py38-cython/bin/python> target/.tox-py38-cython/py38-cython/bin/pip install --retries 10 --exists-action w '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-cython/.tmp/package/1/apache-beam-2.34.0.dev0.zip[test,dataframe]'> (exited with code -15 (SIGTERM)) (exited with code -15)
[0mscripts/run_tox.sh: line 62: 8311 Terminated tox -c tox.ini --recreate -e "$TOX_ENVIRONMENT"
> Task :sdks:python:test-suites:tox:py36:testPy36Cython
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py36/build/srcs/sdks/python/target/.tox-py36-cython/py36-cython/lib/python3.6/site-packages/setuptools/dist.py>:487: UserWarning: Normalizing '2.34.0.dev' to '2.34.0.dev0'
warnings.warn(tmpl.format(**locals()))
error: command 'x86_64-linux-gnu-gcc' terminated by signal 15
[31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py36/build/srcs/sdks/python/target/.tox-py36-cython/py36-cython/bin/python> setup.py build_ext --inplace (exited with code -15 (SIGTERM)) (exited with code -15)
[0m[1mpy36-cython run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py36/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0mscripts/run_tox.sh: line 62: 2988 Terminated tox -c tox.ini --recreate -e "$TOX_ENVIRONMENT"
failed to abort command 'sh'
org.gradle.process.internal.ExecException: A problem occurred waiting for process 'command 'sh'' to complete.
at org.gradle.process.internal.DefaultExecHandle.execExceptionFor(DefaultExecHandle.java:241)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:218)
at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:33)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:365)
at org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84)
... 7 more
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_batch_V2 FAILED
failed to abort command 'sh'
org.gradle.process.internal.ExecException: A problem occurred waiting for process 'command 'sh'' to complete.
at org.gradle.process.internal.DefaultExecHandle.execExceptionFor(DefaultExecHandle.java:241)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:218)
at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:33)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:365)
at org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84)
... 7 more
> Task :sdks:python:test-suites:tox:py36:testPy36Cython FAILED
failed to abort command 'sh'
org.gradle.process.internal.ExecException: A problem occurred waiting for process 'command 'sh'' to complete.
at org.gradle.process.internal.DefaultExecHandle.execExceptionFor(DefaultExecHandle.java:241)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:218)
at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:33)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:365)
at org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84)
... 7 more
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_batch_V2 FAILED
Failed to execute org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable@222f34c5.
java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:33)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
Failed to execute org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable@5d021ccf.
java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:33)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
> Task :sdks:python:test-suites:tox:py37:testPy37Cython
scripts/run_tox.sh: line 62: 26524 Terminated tox -c tox.ini --recreate -e "$TOX_ENVIRONMENT"
Daemon will be stopped at the end of the build after the daemon was no longer found in the daemon registry
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=c0805065-9679-4130-ae1c-e42584182ebb, currentDir=<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 15718
log file: /home/jenkins/.gradle/daemon/6.9.1/daemon-15718.out.log
----- Last 20 lines from daemon log file - daemon-15718.out.log -----
at org.gradle.process.internal.DefaultExecHandle.execExceptionFor(DefaultExecHandle.java:241)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:218)
at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:33)
at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:365)
at org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108)
at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84)
... 7 more
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-0
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PreCommit_Python_Cron #4773
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4773/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4772
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4772/display/redirect>
Changes:
------------------------------------------
[...truncated 1.04 MB...]
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 51.12 seconds ======
[1mpy38-pyarrow-5 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-5: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f9fb0789fd0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f9ff59b3dd8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f9fc02d8780>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-08_05_35_41-12025630442679267772 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output1998105a-e432-4db8-84aa-e03a0c362298.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 962.10 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f319c6a1198>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f319c6a10b8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f31a5278d68>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-08_05_36_32-12969015970264857613 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputdfafc894-23bb-4165-8216-6abcd5326de5.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 952.63 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 21s
107 actionable tasks: 77 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/ty3ixsyqee75k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4771
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4771/display/redirect?page=changes>
Changes:
[Udi Meiri] Release script fixes
------------------------------------------
[...truncated 1.02 MB...]
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 22 warnings in 34.08 seconds ==============
ERROR:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/_pytest/config/__init__.py",> line 447, in _importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/conftest.py')>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/gcp/bigquery.py",> line 341, in <module>
import google.cloud.bigquery_storage_v1 as bq_storage
ModuleNotFoundError: No module named 'google.cloud'
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4979 items / 4950 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 46.65 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-5
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-5 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5>
[0m[1mpy38-pyarrow-5 installdeps: pyarrow>=5,<6
[0m[1mpy38-pyarrow-5 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/.tmp/package/1/apache-beam-2.35.0.dev0.zip>
[0m[1mpy38-pyarrow-5 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/.tmp/package/1/apache-beam-2.35.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.6,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.3,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.5,freezegun==1.1.0,future==0.18.2,greenlet==1.1.2,grpcio==1.41.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.10.0,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.4,packaging==21.0,pandas==1.3.3,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.18.1,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==5.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.3,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.25,tenacity==5.1.5,testcontainers==3.4.2,typing-extensions==3.10.0.2,urllib3==1.26.7,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.13.1>
[0m[1mpy38-pyarrow-5 run-test-pre: PYTHONHASHSEED='2756305051'
[0m[1mpy38-pyarrow-5 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-5 run-test-pre: commands[1] | pip --version
[0mpip 21.2.4 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-5 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-5 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-5 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/.tmp/package/1/apache-beam-2.35.0.dev0.zip>
numpy==1.20.3
pyarrow==5.0.0
[1mpy38-pyarrow-5 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-5 '' '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-5/py38-pyarrow-5/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:569
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5.xml> -
============== 29 passed, 1 skipped, 18 warnings in 33.54 seconds ==============
ERROR:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/_pytest/config/__init__.py",> line 447, in _importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/conftest.py')>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/gcp/bigquery.py",> line 341, in <module>
import google.cloud.bigquery_storage_v1 as bq_storage
ModuleNotFoundError: No module named 'google.cloud'
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-5/py38-pyarrow-5/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4979 items / 4950 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 46.77 seconds ======
[1mpy38-pyarrow-5 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-5: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 57m 12s
107 actionable tasks: 77 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/poii27okrj3j2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4770
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4770/display/redirect?page=changes>
Changes:
[noreply] Revert "[BEAM-12993] Update to Debezium 1.7.0.Final (#15636)"
[kawaigin] Updated screendiff integration test golden screenshots.
[noreply] [BEAM-12769] Few fixes related to Java Class Lookup based cross-language
------------------------------------------
[...truncated 1.06 MB...]
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 46.76 seconds ======
[1mpy38-pyarrow-5 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-5: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f07f5df4400>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f07f5bc3fd0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f07f426da58>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-07_17_38_33-11994387435443414508 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputc0e57f7a-8b29-40e1-a0ce-d79eab5038d0.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 962.18 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fc3ed1999b0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fc3ed199fd0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fc3ed1bf630>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-07_17_38_33-3746461864564763210 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output2acc7fd9-fe54-41a1-97c4-96d624d1d610.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 986.91 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 23s
107 actionable tasks: 77 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/acjz74kja35u2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4769
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4769/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13015] Remove the overhead of SpecMonitoringInfoValidator
[noreply] Minor: Replace generic external.py links in multi-language documentation
------------------------------------------
[...truncated 1.02 MB...]
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 48.91 seconds ======
[1mpy38-pyarrow-5 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-5: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f71abb51748>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f71740f4fd0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f71740f4f98>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-07_11_54_33-3239144228780277585 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output3e222f47-fce6-46cf-b3b9-fcb9d75c6a16.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 942.44 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw1] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7ff24bda4860>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7ff191781e48>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7ff1b2baff28>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-07_11_54_42-11556104484118601575 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output2559aa6c-3ec3-48c2-8911-f67414a6380e.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 984.54 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 2s
107 actionable tasks: 77 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/dk4wp4bv3cyb4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4768
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4768/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f4ca81be080>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f4cddc41ba8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f4ca8573b00>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-07_05_36_54-14478921563839662238 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output2cc128bd-fa3a-4575-8997-0ce191aecd7a.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 957.16 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-5
.... [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 49.64 seconds ======
[1mpy38-pyarrow-5 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-5: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fe114791f98>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fe114791f60>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fe114791f28>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-07_05_37_54-3847012002925252055 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputd2cd3dc1-7c14-4e31-b2ce-f1a2a44ebb72.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 951.68 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 4s
107 actionable tasks: 77 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/rp7pps3jkxgvm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4767
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4767/display/redirect?page=changes>
Changes:
[david.prieto] [BEAM-12950] Not delete orphaned files to avoid missing events
[david.prieto] [BEAM-12950] Add Bug fix description to CHANGES.md
[david.prieto] [BEAM-12950] fix linter issues
[david.prieto] [BEAN-12950] Skip unit test
------------------------------------------
[...truncated 993.38 KB...]
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f20940c8978>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f20940c8080>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f2094ca6b00>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-06_23_38_53-17169888107748814544 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output02842276-07cb-4d05-b8ce-6addfb0e1871.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 967.97 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-5
. [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f4d30049f98>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f4d30049eb8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f4dcaf2e518>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super().run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:68: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-10-06_23_39_51-9229575673683784845 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output04ff8dc5-a50d-4323-b6c7-15bdd2023178.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 952.81 seconds ==========================
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-5
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 53.11 seconds ======
[1mpy38-pyarrow-5 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-5: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 52m 12s
107 actionable tasks: 78 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/wyh4dhd3iaozg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4766
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4766/display/redirect?page=changes>
Changes:
[stefan.istrate] Fix "too many pings" errors.
[stefan.istrate] Increase keepalive timeout to 5 minutes.
[Robert Bradshaw] Dead letter option.
[Robert Bradshaw] Guard setup.py logic with __main__ condition.
[stefan.istrate] Fix yapf complaints.
[aydar.zaynutdinov] [BEAM-12969] [Playground]
[Robert Bradshaw] Add the ability to use subprocesses with the dead letter queue.
[Robert Bradshaw] Support multi-output DoFns.
[noreply] Merge pull request #15510 from [BEAM-12883] Add coder for
[Robert Bradshaw] multi-output fix
[Robert Bradshaw] Add thresholding to dead letter pattern.
[Robert Bradshaw] treshold test fixes
[Robert Bradshaw] Better naming, documentation.
[noreply] [BEAM-12482] Ensure that we ignore schema update options when loading
[Kyle Weaver] Moving to 2.35.0-SNAPSHOT on master branch.
[Kyle Weaver] Add 2.35.0 section to changelog.
[noreply] Fix email links in the contact page
------------------------------------------
[...truncated 1.01 MB...]
timeout method: signal
timeout func_only: False
collected 4979 items / 4950 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 50.75 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-5
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-5 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5>
[0m[1mpy38-pyarrow-5 installdeps: pyarrow>=5,<6
[0m[1mpy38-pyarrow-5 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/.tmp/package/1/apache-beam-2.35.0.dev0.zip>
[0m[1mpy38-pyarrow-5 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/.tmp/package/1/apache-beam-2.35.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.6,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.2,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.5,freezegun==1.1.0,future==0.18.2,greenlet==1.1.2,grpcio==1.41.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.10.0,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.4,packaging==21.0,pandas==1.3.3,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.18.1,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==5.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.3,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.25,tenacity==5.1.5,testcontainers==3.4.2,typing-extensions==3.10.0.2,urllib3==1.26.7,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.13.1>
[0m[1mpy38-pyarrow-5 run-test-pre: PYTHONHASHSEED='3403294354'
[0m[1mpy38-pyarrow-5 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-5 run-test-pre: commands[1] | pip --version
[0mpip 21.2.4 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-5 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-5 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-5 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/.tmp/package/1/apache-beam-2.35.0.dev0.zip>
numpy==1.20.3
pyarrow==5.0.0
[1mpy38-pyarrow-5 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-5 '' '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-5/py38-pyarrow-5/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:569
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5.xml> -
============== 29 passed, 1 skipped, 20 warnings in 36.60 seconds ==============
ERROR:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/_pytest/config/__init__.py",> line 447, in _importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/conftest.py')>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/gcp/bigquery.py",> line 341, in <module>
import google.cloud.bigquery_storage_v1 as bq_storage
ModuleNotFoundError: No module named 'google.cloud'
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-5/py38-pyarrow-5/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4979 items / 4950 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-5/py38-pyarrow-5/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:569: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-5_no_xdist.xml> -
====== 29 passed, 3 skipped, 4950 deselected, 7 warnings in 49.44 seconds ======
[1mpy38-pyarrow-5 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-5: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_batch'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_batch'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 80
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 47m 43s
107 actionable tasks: 79 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/7q3qcqgnolqn2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org