You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/08/09 06:52:48 UTC
Build failed in Jenkins: beam_PreCommit_Python_Cron #4529
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4529/display/redirect>
Changes:
------------------------------------------
[...truncated 954.77 KB...]
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 46.68 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fc4a00d5da0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fc53a1d9908>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fc4a14e1d68>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-08_23_36_44-4301866826313689768 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output652030d4-6225-4aac-84ba-76cf324e7da1.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 941.52 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f736456bf28>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f736456bb38>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f736456bb00>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-08_23_37_02-3925161278007750736 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputafc734d5-5285-40ac-9033-dd3a75e9f2bf.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 957.22 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py36:testPy36Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 50m 19s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/7vlrovfct4o32
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PreCommit_Python_Cron #4564
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4564/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4563
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4563/display/redirect>
Changes:
------------------------------------------
[...truncated 980.88 KB...]
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-3/py38-pyarrow-3/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ..................
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 passed in 645.28 seconds ==========================
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-3
......
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 passed in 651.98 seconds ==========================
> Task :sdks:python:test-suites:dataflow:preCommitIT_V2
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-3
.... [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 47.88 seconds ======
[1mpy38-pyarrow-3 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-3: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.3.2,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.2,typing-extensions==3.10.0.0,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='525346215'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.1.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 24 warnings in 34.78 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 47.32 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 5s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/ecbctcrpyfskc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4562
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4562/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11088] TestStream utility and testing improvements (#15320)
------------------------------------------
[...truncated 963.90 KB...]
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3.xml> -
============== 29 passed, 1 skipped, 20 warnings in 43.49 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-3/py38-pyarrow-3/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 passed in 702.84 seconds ==========================
> Task :sdks:python:test-suites:dataflow:preCommitIT_V2
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-3
................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 53.79 seconds ======
[1mpy38-pyarrow-3 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-3: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.3.2,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.2,typing-extensions==3.10.0.0,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='2890928696'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.2.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 20 warnings in 43.61 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 52.88 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 54m 28s
106 actionable tasks: 78 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/kdfvw46icun5q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4561/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12609] Make type parameter part of PushdownProjector interface.
[pascal.gillet] [BEAM-12479] Fixes UnsupportedOperationException
[baeminbo] [BEAM-12504] Make CreateTransaction wait on input signal
[Robert Burke] redundant build
------------------------------------------
[...truncated 979.22 KB...]
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3.xml> -
============== 29 passed, 1 skipped, 24 warnings in 34.45 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-3/py38-pyarrow-3/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 48.56 seconds ======
[1mpy38-pyarrow-3 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-3: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.3.2,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.2,typing-extensions==3.10.0.0,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='3015897274'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.1.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 24 warnings in 34.88 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 46.86 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 passed in 676.26 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 passed in 699.45 seconds ==========================
> Task :sdks:python:test-suites:dataflow:preCommitIT_V2
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 47m 19s
106 actionable tasks: 78 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/6pnlt4v7d7bxo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4560/display/redirect?page=changes>
Changes:
[noreply] [BEAM-7745] Avoid uncached state fetches for streaming side-inputs
------------------------------------------
[...truncated 1020.15 KB...]
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 50.83 seconds ======
[1mpy38-pyarrow-3 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-3: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_batch_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 passed in 654.86 seconds ==========================
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/build/apache-beam.tar.gz> --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --streaming --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.33.0-SNAPSHOT.jar> --experiments=use_runner_v2 --experiments=shuffle_mode=appliance --enable_streaming_engine
>>> pytest options: apache_beam/examples/streaming_wordcount_it_test.py::StreamingWordCountIT::test_streaming_wordcount_it --capture=no --numprocesses=2 --timeout=1800
>>> collect markers:
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 1800.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I
gw0 [1] / gw1 [1]
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.3.2,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.2,typing-extensions==3.10.0.0,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='3452408952'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.1.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 24 warnings in 36.72 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 50.31 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 passed in 669.80 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 passed in 705.88 seconds ==========================
> Task :sdks:python:test-suites:dataflow:preCommitIT_V2
FAILURE: Build completed with 5 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 10s
106 actionable tasks: 77 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/ab6esffr4neys
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4559/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-12619] Swap LinkedBlockingQueue to ArrayBlockingQueue for minor
[Luke Cwik] fixup! Fix spotbugs warning
[Jeremy Quinn] Add AWS services as a runtime dependency to support S3
[Andrew Pilloud] [BEAM-12759] ORDER BY then SELECT
------------------------------------------
[...truncated 1019.95 KB...]
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 18 warnings in 39.54 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ........................
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 passed in 690.15 seconds ==========================
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
.... [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 51.70 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw1] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f48b86d2908>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f48b86d2780>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f48b86d2748>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-16_11_59_01-2767531113701067275 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputc9575a4c-fc85-4874-9722-e55ac8136335.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 942.88 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 5 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 47m 35s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/6wjui7lzvzffo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4558/display/redirect>
Changes:
------------------------------------------
[...truncated 1.05 MB...]
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-16_05_38_45-14807344086465079261 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputa00d93f1-3d00-4189-a005-1c44bec0f14b.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 944.61 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.3.2,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.2,typing-extensions==3.10.0.0,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='1048564202'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.1.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 24 warnings in 35.89 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 49.37 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 53m 37s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/oz3k5pyorcgqu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4557
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4557/display/redirect>
Changes:
------------------------------------------
[...truncated 1.02 MB...]
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fa268f1d7f0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fa268f1d400>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fa268f1d3c8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_23_35_29-12439228726966791233 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output17d79ad9-7ad9-4cea-8f7c-b3e2e3fea9b4.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 955.07 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f3b651ee860>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f3acc092cf8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f3acc42c908>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_23_35_32-2895038303767788707 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output1d2a077d-07bc-46b1-b241-6af649f17bbf.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 946.51 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 48m 26s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/ux53gw3myblwa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4556
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4556/display/redirect>
Changes:
------------------------------------------
[...truncated 1005.61 KB...]
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f8a9c44d400>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f8a9c44d320>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f8a9c44d2e8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_17_35_55-2308941972274573784 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output62bbeb3b-f88b-4450-9e3b-2a0cad98284b.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 975.80 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f3558126898>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f358dd8fe10>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f3558d166a0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_17_36_51-5994842429834711271 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output176c13a7-33b5-471e-b88b-7bfddcb5a7df.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 951.81 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 50m 11s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/cowo3p4ujkxce
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4555
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4555/display/redirect>
Changes:
------------------------------------------
[...truncated 979.55 KB...]
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fbcbc665908>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fbcbc665518>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fbcbf26c780>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_11_37_08-10009109692022599270 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputbe60fb25-dc8f-43da-a98a-6b01ef1e09d3.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 961.41 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7ff3d80970b8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7ff4731e8518>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7ff3d8097080>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_11_38_05-5428781077632820941 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output77d6e3b1-cb16-4322-abfe-24f87b7b6fd0.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 946.91 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 6 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
5: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
6: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 6s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/6z62j6ssve7eo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4554
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4554/display/redirect>
Changes:
------------------------------------------
[...truncated 996.24 KB...]
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 53.34 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f26331afcf8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f25fc46a400>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f25fc46a3c8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_05_35_26-452522900742146525 after 361 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output085c86d1-80e7-493b-af20-103dbb2db813.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 951.21 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f9018666278>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f9018666198>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f9018666160>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-15_05_35_35-15906105398263307150 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output9ed4d30e-724b-49d9-b294-ade91b044cc7.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 955.94 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py36:testPy36Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 48m 52s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/sv2x2evsfueqq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4553
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4553/display/redirect>
Changes:
------------------------------------------
[...truncated 1006.28 KB...]
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 48.26 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f8ef424a160>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f8ef424a080>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f8ef424a048>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_23_35_19-14855913448484727983 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputfae38ee5-29eb-4217-88eb-1495e4ee7e5b.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 972.53 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw1] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7ff4c40e6e48>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7ff4c40e66d8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7ff4c54aa6a0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_23_35_57-991178715758474392 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputb9fb2d67-0fae-452a-93d7-bf3317b2dd44.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 958.41 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py36:testPy36Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 16s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/bgmge7iqlzh54
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4552
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4552/display/redirect>
Changes:
------------------------------------------
[...truncated 993.82 KB...]
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 47.89 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f8f84501ef0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f8f845012e8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f8f845012b0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_17_35_12-3247587248591132213 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputc211a7b7-de0b-4741-b434-732429d01636.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 952.27 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f6cec35b1d0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f6cec35b0f0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f6cec35b0b8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_17_35_40-13224962063562997416 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output9f991d26-7bea-45dd-9dd2-63312fa1ac36.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 958.41 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38CloudCoverage'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 7s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/s5mnywmefwj3g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4551
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4551/display/redirect>
Changes:
------------------------------------------
[...truncated 1.04 MB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 51.89 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fe4504d5ef0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fe4504d52e8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fe4504d52b0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_11_36_56-15066243365905492869 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output8f29dc52-4dc6-4f54-af6a-bff04e5136e0.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 940.18 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f875c39b1d0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f875c39b0f0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f875c39b0b8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_11_36_29-3401236650577587403 after 363 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputb398ab4f-5fe1-4a04-a8c9-fca1f112f37e.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 968.07 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 55s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/w7bwekh3mnz7e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4550
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4550/display/redirect>
Changes:
------------------------------------------
[...truncated 1.05 MB...]
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7ffac4766208>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7ffb6d831400>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7ffb6d827ef0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_05_38_05-4646719249153619208 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output8bdb9c02-f6ce-4fa6-9ed1-9eed1afe598a.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 964.74 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f9068739cf8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f9068739908>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f90335bf5f8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-14_05_38_50-5640448151978939504 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output85d2de9b-c565-4770-b2f0-8aa780467cf1.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 943.49 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py37:testPy37Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py36:testPy36Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 35s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/67lexd3wa4qqs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4549
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4549/display/redirect>
Changes:
------------------------------------------
[...truncated 1023.07 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 47.38 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f8b3975e438>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f8b8f4ecd30>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f8b3975e400>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-13_23_33_48-1143320599239311611 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputa272cb65-be2c-4e71-ae80-1b35e210100a.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 938.91 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f668407af98>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f67209e7588>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f668407a128>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-13_23_34_51-3241367520456588446 after 362 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output7554f5d3-e033-4d0a-adea-0b58d98495e6.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 966.64 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 48m 34s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/eoltigekxmgve
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4548
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4548/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12757]: Remove call for Json Dag in Portable PipelineRunner as
[noreply] fix confusing function def (#15120)
------------------------------------------
[...truncated 1.00 MB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 49.22 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f4ec4294d30>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f4ef7fa8e48>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f4ec422d438>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-13_17_35_48-8714458526480605463 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output76f2bca1-911b-4c4d-95ec-2277e2df7412.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 943.91 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fbd6020f198>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fbd6020f0b8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fbd6020f080>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-13_17_36_39-7515110842330956469 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outpute4d9b1b0-05a7-4712-bf7d-deb812b32d52.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 958.75 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 58s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/2hptluohbulru
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4547/display/redirect>
Changes:
------------------------------------------
[...truncated 1.02 MB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 50.90 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fa70c0f1f28>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fa70c0f1320>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fa70dce4630>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-13_11_40_25-17060811835384075073 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputbfaa76d6-7822-49bb-99a8-90c4799d207d.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 943.42 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fc81406aeb8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fc81406add8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fc81406ada0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-13_11_40_46-2747841997973101800 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output85ab0936-358f-45c5-9540-d43c66f610ca.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 962.72 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 54m 15s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/hursgqwxi3tns
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4546
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4546/display/redirect>
Changes:
------------------------------------------
[...truncated 1005.72 KB...]
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-13_05_41_21-458288695361669474 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output4bceb561-6af4-4b4e-89a3-06de4221cc5e.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 956.19 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-3
......................... [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 49.53 seconds ======
[1mpy38-pyarrow-3 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-3: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.3.1,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.1,typing-extensions==3.10.0.0,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.1,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='2950817634'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.2.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============== 29 passed, 1 skipped, 22 warnings in 36.16 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 48.53 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 56m 42s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/n67bl7cl46k5e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4545
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4545/display/redirect>
Changes:
------------------------------------------
[...truncated 1.01 MB...]
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 50.09 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f8eb3d27908>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f8eb3d274e0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f8e5579e3c8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-12_23_36_27-12509782872513738688 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputac4ca9fe-4fba-492c-9506-2c536e59504f.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 955.34 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f46e80b34a8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f46e80b33c8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f46e80b3390>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-12_23_36_55-4539851383052891720 after 363 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output16153f02-cb9c-4900-9a38-575dc47a129a.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 949.68 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 12s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/7jps54vy2a42e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4544
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4544/display/redirect?page=changes>
Changes:
[noreply] Loosen typing extensions requirement
[Kyle Weaver] [BEAM-12719] ZetaSQL docs: replace out of date table with link to
[ryanthompson591] Add retry_gcs_file_copy when 500 errros happen from server
[noreply] Merge pull request #15265 from [BEAM-12545] Extend FileSink by
[noreply] [BEAM-12453]: Add interface to access I/O topic information for a Samza
[noreply] [BEAM-10212] Guard state caching with experiment (#15319)
------------------------------------------
[...truncated 1.01 MB...]
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3.xml> -
============= 29 passed, 1 skipped, 18 warnings in 198.43 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-3/py38-pyarrow-3/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3_no_xdist.xml> -
===== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 244.08 seconds ======
[1mpy38-pyarrow-3 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-3: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.3.1,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.1,typing-extensions==3.10.0.0,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.0,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='2055372742'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.1.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============= 29 passed, 1 skipped, 16 warnings in 187.73 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4900 items / 4871 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
===== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 141.25 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 11m 8s
106 actionable tasks: 78 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/k7x32rnkn56gi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4543
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4543/display/redirect?page=changes>
Changes:
[bsindhwani] Python Kata - Event Time Triggers
[bsindhwani] Python Kata - Event Time Triggers - Added license to task-info.yaml
[bsindhwani] Python Kata - Event Time Triggers - refactored the accumulation mode
[bsindhwani] Python Kata - Event Time Triggers - task description corrected to
[bsindhwani] Python Kata - Event Time Triggers - added streaming options
[bsindhwani] Python Kata - Event Time Triggers - add license text in task-info.yaml
[bsindhwani] Python Kata - Early Triggers
[bsindhwani] Python Kata - Event Time Triggers & Early Triggers - task.md - corrected
[bsindhwani] Python Kata - Window Accumulation Mode
[bsindhwani] Python Kata - Event Time Triggers - removed the external transform
[bsindhwani] Python Kata - added license to course-info.yaml
[Robert Burke] [BEAM-11779] Remove appliance restriction.
[noreply] [BEAM-6374] Emit PCollection metrics from GoSDK (#15289)
[noreply] Merge pull request #15165 from [BEAM-12593] Verify DataFrame API on
------------------------------------------
[...truncated 1006.26 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4871 deselected, 7 warnings in 50.45 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f205000dc50>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f20ec0f34a8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f205000dc18>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-12_11_35_37-10497948288583496027 after 365 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output23156204-f3b0-4cc5-94cb-52be6e7dd5ea.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 957.68 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f784ee55a20>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f7818164a58>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f7818164a20>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-12_11_37_15-1100093349174814580 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output59bc3fa6-04aa-4088-9df3-5618e558705a.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 950.36 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 58s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/p2uzkh3wvgtea
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4542
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4542/display/redirect>
Changes:
------------------------------------------
[...truncated 974.57 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4870 deselected, 7 warnings in 48.02 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw1] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f86fc27de10>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f86fc27dd30>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f86fc27dcf8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-12_05_35_24-17926447789040982966 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output4ca5dd4e-c030-4c9d-b140-025c1deed01b.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 956.16 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f24b478feb8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f24b478fdd8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f24b478fda0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-12_05_36_26-3700284783593632165 after 361 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output2fe0cf20-fe77-4fe9-9fa4-74095f0d651e.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 965.04 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 54s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/wsfld2hu3hbho
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4541
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4541/display/redirect?page=changes>
Changes:
[noreply] [GoSDK Infra] Limit simultaneous tests binaries to 3. (#15321)
------------------------------------------
[...truncated 1.02 MB...]
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:534
apache_beam/io/fileio.py:534
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:534: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3.xml> -
============= 29 passed, 1 skipped, 16 warnings in 142.74 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-3/py38-pyarrow-3/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4899 items / 4870 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-3/py38-pyarrow-3/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:534: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-3_no_xdist.xml> -
===== 29 passed, 3 skipped, 4870 deselected, 7 warnings in 220.33 seconds ======
[1mpy38-pyarrow-3 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-3: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
[1mGLOB sdist-make: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/setup.py>
[0m[1mpy38-pyarrow-4 create: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4>
[0m[1mpy38-pyarrow-4 installdeps: pyarrow>=4,<5
[0m[1mpy38-pyarrow-4 inst: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
[0m[1mpy38-pyarrow-4 installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,charset-normalizer==2.0.4,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,docker==5.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.4.4,freezegun==1.1.0,future==0.18.2,greenlet==1.1.1,grpcio==1.39.0,hdfs==2.6.0,httplib2==0.19.1,idna==3.2,mock==2.0.0,more-itertools==8.8.0,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,orjson==3.6.1,packaging==21.0,pandas==1.2.5,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.9.1,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.0,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2021.1,PyYAML==5.4.1,requests==2.26.0,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.22,tenacity==5.1.5,testcontainers==3.4.1,typing-extensions==3.7.4.3,urllib3==1.26.6,wcwidth==0.2.5,websocket-client==1.2.0,wrapt==1.12.1>
[0m[1mpy38-pyarrow-4 run-test-pre: PYTHONHASHSEED='249575482'
[0m[1mpy38-pyarrow-4 run-test-pre: commands[0] | python --version
[0mPython 3.8.5
[1mpy38-pyarrow-4 run-test-pre: commands[1] | pip --version
[0mpip 21.1.3 from <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/pip> (python 3.8)
[1mpy38-pyarrow-4 run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mpy38-pyarrow-4 run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mpy38-pyarrow-4 run-test: commands[0] | /bin/sh -c 'pip freeze | grep -E '"'"'(pyarrow|numpy)'"'"''
[0mapache-beam @ file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/.tmp/package/1/apache-beam-2.33.0.dev0.zip>
numpy==1.20.3
pyarrow==4.0.1
[1mpy38-pyarrow-4 run-test: commands[1] | <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh> py38-pyarrow-4 '-m uses_pyarrow'
[0m============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I
gw0 [29] / gw1 [29] / gw2 [29] / gw3 [29] / gw4 [29] / gw5 [29]
............................. [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
apache_beam/io/parquetio_test.py:419
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py:380
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py:370
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/dataframe/io.py:566
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/io/fileio.py:534
apache_beam/io/fileio.py:534
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:534: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4.xml> -
============= 29 passed, 1 skipped, 20 warnings in 169.05 seconds ==============
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
cachedir: target/.tox-py38-pyarrow-4/py38-pyarrow-4/.pytest_cache
rootdir: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4899 items / 4870 deselected / 3 skipped / 26 selected
apache_beam/dataframe/io_test.py . [ 3%]
apache_beam/io/parquetio_test.py ............................ [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:534: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:419: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:380: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:370: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
===== 29 passed, 3 skipped, 4870 deselected, 7 warnings in 208.86 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py36:testPy36Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py38:testPy38Cython'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 2m 42s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/ccbbkupegbv4s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4540
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4540/display/redirect?page=changes>
Changes:
[marwant] Move misplaced cloud healthcare update info line from template to
[Robert Burke] [BEAM-12738][GoSDK] Fix Dataflow Job URL
[eugene.nikolayev] Fix docs typos in contrib guide.
[noreply] Merge pull request #15126 from [BEAM-12665] Add option in ReadAll
[noreply] Update ptransform.py error message (#15286)
------------------------------------------
[...truncated 947.13 KB...]
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 29 passed, 3 skipped, 4870 deselected, 7 warnings in 48.32 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f9a3c283588>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f9a3c283400>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f99a351c5c0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-11_17_36_12-5480933475249157816 after 365 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputc4e1cf10-69d0-47cc-9f3b-d754c09c2e03.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 968.05 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fe870751d68>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fe8b688dbe0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fe878b195c0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-11_17_36_31-12897241504821844767 after 361 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputa1d18a87-4d3f-4754-aced-1263fe339829.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 958.91 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:py36:testPy36Cloud'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 50s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/o6jmsa6ai2kv2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4539
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4539/display/redirect>
Changes:
------------------------------------------
[...truncated 946.79 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 61.85 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f834c339c88>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f834c3397b8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f834c339940>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-11_11_38_40-12047258655293325527 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output99a39f49-106e-4e10-a020-19513005b96c.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 945.28 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fd10c192898>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fd10c192780>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fd1a62e3438>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-11_11_39_58-8002974064291293977 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputef5c405c-110a-44ce-b157-f7d142778f41.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 955.73 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 52m 26s
106 actionable tasks: 78 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/vmwzodqma7brw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4538
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4538/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-4 (beam) in workspace <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git --version # 'git version 2.7.4'
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b9b4c6e1f142d3fcb93f6a4458e2a06282ab9ee8 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b9b4c6e1f142d3fcb93f6a4458e2a06282ab9ee8 # timeout=10
Commit message: "Merge pull request #15296 from andyxiexu/python-sdk"
> git rev-list --no-walk b9b4c6e1f142d3fcb93f6a4458e2a06282ab9ee8 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib
[EnvInject] - Variables injected successfully.
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/gradlew> --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :pythonPreCommit
Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy
> Task :buildSrc:spotlessGroovy FAILED
> Task :buildSrc:spotlessGroovyGradle FROM-CACHE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins FROM-CACHE
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildSrc:spotlessGroovy'.
> Failed to load cache entry for task ':buildSrc:spotlessGroovy'
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 3s
Publishing build scan...
https://gradle.com/s/3ammsxnwp6fw6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4537
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4537/display/redirect>
Changes:
------------------------------------------
[...truncated 952.73 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 51.28 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw1] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fc4b85dec50>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fc4f66b5e10>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fc4b8965240>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_23_36_50-5599074931511338999 after 362 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputa374f055-d476-400b-9dc3-fdfb691fd8b4.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 949.89 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f39b877ccf8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f3a6a8d4748>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f39d1bc2d68>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_23_38_27-5083835316726425762 after 361 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output9afb1f6a-8352-480f-b3dc-823856893d0a.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 959.43 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 27s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/mg6734f3igb5i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4536
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4536/display/redirect?page=changes>
Changes:
[andyxu] Add support for python sdk container to turn on google cloud profiler
[andyxu] format file
[marwant] Update Google Cloud Healthcare API version from v1beta1 to GA
[zyichi] Update beam dataflow python container versions.
------------------------------------------
[...truncated 972.40 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 47.03 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw1] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fe8c847cba8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fe8c847c748>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fe8c847c898>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_17_40_41-14687195038530836389 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputfd0fc177-08f3-4e3b-94ad-8498283e4141.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 942.37 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f135c73edd8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f135c73ecf8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f135c73ecc0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_17_41_32-11127941550024940732 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output6f420abe-42db-418e-8c98-0e41ae33cae6.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 970.91 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 23s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/7gwh4rke3g3ae
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4535
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4535/display/redirect?page=changes>
Changes:
[chamikaramj] Fix typo
[noreply] [BEAM-12403][BEAM-12348] Add User State Support for Portable SamzaRunner
------------------------------------------
[...truncated 926.69 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 49.35 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f2c405b2d30>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f2c405b2b38>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f2c405b2b00>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_11_40_25-3188716562485011862 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputac095901-8140-42c0-bb8f-c35234e0fd38.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 949.53 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f2e1dfcbba8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f2e1dfcb748>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f2e2736b2e8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_11_41_07-2615666025474055681 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output952cdfe9-6d61-4f96-bd8a-f777f8b4dc98.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 953.34 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 17s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/tx5mmgt5lhlcu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4534
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4534/display/redirect>
Changes:
------------------------------------------
[...truncated 942.46 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 47.83 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f4bb40c6b70>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f4bb40c6860>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f4bf29fe8d0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_05_38_24-3919780683032473456 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output796e3003-8fbc-4ff1-843a-e414049f7e83.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 952.12 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fa1743b1c88>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fa1743b1a90>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fa20f5196a0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-10_05_39_38-5177526076263024803 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output427b89bb-9b3e-4248-9834-02dd3358bf33.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 962.16 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 52m 13s
106 actionable tasks: 78 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/igwg2y6tkxde4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4533
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4533/display/redirect?page=changes>
Changes:
[heejong] [BEAM-12716] Update XLang Kafka taxi example to use BigQuery sink
[heejong] add comment
------------------------------------------
[...truncated 930.91 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 48.78 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f7cfc4bcc50>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f7cfc4bc748>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f7cfc4bc940>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_23_37_42-17943723580748026325 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output851c1761-37cb-4340-9637-7e30e1bab474.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 954.12 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fe901f40cf8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fe9c4084438>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fe9c40847b8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_23_38_04-16352191386617522697 after 363 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output5c4a8284-40d0-4514-b8fd-f81ea7a1618b.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 949.25 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 17s
106 actionable tasks: 78 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/rpniknxqh66jg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4532
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4532/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #15298 from [BEAM-12729] Suppress Avro Runtime
[noreply] Refactor _RemoveDuplicates to use counter state (#15242)
------------------------------------------
[...truncated 963.16 KB...]
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7fcc94240e48>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7fcc94240d68>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7fcc94240d30>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_17_51_32-12523917916217857872 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_outputa97db51b-e801-44a6-85ab-9965d575a272.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 952.72 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:tox:py38:testPy38pyarrow-4
.... [100%]
=============================== warnings summary ===============================
target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-pyarrow-4/py38-pyarrow-4/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/dataframe/io.py>:566: FutureWarning: WriteToFiles is experimental.
return pcoll | fileio.WriteToFiles(
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
apache_beam/dataframe/io_test.py::IOTest::test_read_write_parquet
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/fileio.py>:534: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/parquetio_test.py::TestParquet::test_int96_type_conversion
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:416: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
for batch in orig.to_batches(chunksize=20)
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 61.71 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f8dd4f3c8d0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f8dd4f3c6d8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f8dd4f3c6a0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_17_51_55-14387039196068735823 after 360 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output52c3c4b0-bfd6-4c54-9929-22ff145beb89.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 956.31 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 2s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/3q3zvfdnkitc6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4531
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4531/display/redirect>
Changes:
------------------------------------------
[...truncated 964.05 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 52.28 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f048c5f3d30>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f0525756518>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f048c9f3550>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_11_37_43-5366234889855503110 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output1e4ec2c7-6cf5-4b71-b293-6d0c21ebd022.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 949.01 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7efd16084be0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7efd16084cf8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7efcd8370390>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_11_38_05-1472262083655470551 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output273a31c8-f60d-400a-b075-d86774de1875.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 948.85 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 14s
106 actionable tasks: 76 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/5lpdqrdepk7eq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PreCommit_Python_Cron #4530
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/4530/display/redirect>
Changes:
------------------------------------------
[...truncated 961.00 KB...]
apache_beam/io/parquetio_test.py::TestParquet::test_read_with_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:377: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
apache_beam/io/parquetio_test.py::TestParquet::test_read_without_splitting_multiple_row_group
<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/apache_beam/io/parquetio_test.py>:367: FutureWarning: The parameter chunksize is deprecated for pyarrow.Table.to_batches as of 0.15, please use the parameter max_chunksize instead
pa.Table.from_batches([batch]) for batch in self._records_as_arrow(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-pyarrow-4_no_xdist.xml> -
====== 28 passed, 3 skipped, 4866 deselected, 7 warnings in 46.43 seconds ======
[1mpy38-pyarrow-4 run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary ____________________________________
[32m py38-pyarrow-4: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:tox:py38:preCommitPy38
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f7cc06cbcf8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f7cc06cb860>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f7cd0a643c8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_05_38_06-5045974633597140785 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output82025ca3-9db9-4e58-99be-86a65d7a75c1.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py37.xml> -
========================== 1 failed in 951.33 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2 FAILED
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2
F
=================================== FAILURES ===================================
_______________ StreamingWordCountIT.test_streaming_wordcount_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT testMethod=test_streaming_wordcount_it>
@pytest.mark.it_postcommit
def test_streaming_wordcount_it(self):
# Build expected dataset.
expected_msg = [('%d: 1' % num).encode('utf-8')
for num in range(DEFAULT_INPUT_NUMBERS)]
# Set extra options to the pipeline for test purpose
state_verifier = PipelineStateMatcher(PipelineState.RUNNING)
pubsub_msg_verifier = PubSubMessageMatcher(
self.project, self.output_sub.name, expected_msg, timeout=400)
extra_opts = {
'input_subscription': self.input_sub.name,
'output_topic': self.output_topic.name,
'wait_until_finish_duration': WAIT_UNTIL_FINISH_DURATION,
'on_success_matcher': all_of(state_verifier, pubsub_msg_verifier)
}
# Generate input data and inject to PubSub.
self._inject_numbers(self.input_topic, DEFAULT_INPUT_NUMBERS)
# Get pipeline options from command argument: --test-pipeline-options,
# and start pipeline job by calling pipeline main function.
streaming_wordcount.run(
self.test_pipeline.get_full_options_as_args(**extra_opts),
> save_main_session=False)
apache_beam/examples/streaming_wordcount_it_test.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/streaming_wordcount.py:103: in run
output | beam.io.WriteToPubSub(known_args.output_topic)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f196c00dd30>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f196c00db38>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f196c00db00>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: RUNNING and Expected 500 messages.)
E but: Expected 500 messages. Got 0 messages. Diffs (item, count):
E Expected but not in actual: dict_items([(b'0: 1', 1), (b'1: 1', 1), (b'2: 1', 1), (b'3: 1', 1), (b'4: 1', 1), (b'5: 1', 1), (b'6: 1', 1), (b'7: 1', 1), (b'8: 1', 1), (b'9: 1', 1), (b'10: 1', 1), (b'11: 1', 1), (b'12: 1', 1), (b'13: 1', 1), (b'14: 1', 1), (b'15: 1', 1), (b'16: 1', 1), (b'17: 1', 1), (b'18: 1', 1), (b'19: 1', 1), (b'20: 1', 1), (b'21: 1', 1), (b'22: 1', 1), (b'23: 1', 1), (b'24: 1', 1), (b'25: 1', 1), (b'26: 1', 1), (b'27: 1', 1), (b'28: 1', 1), (b'29: 1', 1), (b'30: 1', 1), (b'31: 1', 1), (b'32: 1', 1), (b'33: 1', 1), (b'34: 1', 1), (b'35: 1', 1), (b'36: 1', 1), (b'37: 1', 1), (b'38: 1', 1), (b'39: 1', 1), (b'40: 1', 1), (b'41: 1', 1), (b'42: 1', 1), (b'43: 1', 1), (b'44: 1', 1), (b'45: 1', 1), (b'46: 1', 1), (b'47: 1', 1), (b'48: 1', 1), (b'49: 1', 1), (b'50: 1', 1), (b'51: 1', 1), (b'52: 1', 1), (b'53: 1', 1), (b'54: 1', 1), (b'55: 1', 1), (b'56: 1', 1), (b'57: 1', 1), (b'58: 1', 1), (b'59: 1', 1), (b'60: 1', 1), (b'61: 1', 1), (b'62: 1', 1), (b'63: 1', 1), (b'64: 1', 1), (b'65: 1', 1), (b'66: 1', 1), (b'67: 1', 1), (b'68: 1', 1), (b'69: 1', 1), (b'70: 1', 1), (b'71: 1', 1), (b'72: 1', 1), (b'73: 1', 1), (b'74: 1', 1), (b'75: 1', 1), (b'76: 1', 1), (b'77: 1', 1), (b'78: 1', 1), (b'79: 1', 1), (b'80: 1', 1), (b'81: 1', 1), (b'82: 1', 1), (b'83: 1', 1), (b'84: 1', 1), (b'85: 1', 1), (b'86: 1', 1), (b'87: 1', 1), (b'88: 1', 1), (b'89: 1', 1), (b'90: 1', 1), (b'91: 1', 1), (b'92: 1', 1), (b'93: 1', 1), (b'94: 1', 1), (b'95: 1', 1), (b'96: 1', 1), (b'97: 1', 1), (b'98: 1', 1), (b'99: 1', 1), (b'100: 1', 1), (b'101: 1', 1), (b'102: 1', 1), (b'103: 1', 1), (b'104: 1', 1), (b'105: 1', 1), (b'106: 1', 1), (b'107: 1', 1), (b'108: 1', 1), (b'109: 1', 1), (b'110: 1', 1), (b'111: 1', 1), (b'112: 1', 1), (b'113: 1', 1), (b'114: 1', 1), (b'115: 1', 1), (b'116: 1', 1), (b'117: 1', 1), (b'118: 1', 1), (b'119: 1', 1), (b'120: 1', 1), (b'121: 1', 1), (b'122: 1', 1), (b'123: 1', 1), (b'124: 1', 1), (b'125: 1', 1), (b'126: 1', 1), (b'127: 1', 1), (b'128: 1', 1), (b'129: 1', 1), (b'130: 1', 1), (b'131: 1', 1), (b'132: 1', 1), (b'133: 1', 1), (b'134: 1', 1), (b'135: 1', 1), (b'136: 1', 1), (b'137: 1', 1), (b'138: 1', 1), (b'139: 1', 1), (b'140: 1', 1), (b'141: 1', 1), (b'142: 1', 1), (b'143: 1', 1), (b'144: 1', 1), (b'145: 1', 1), (b'146: 1', 1), (b'147: 1', 1), (b'148: 1', 1), (b'149: 1', 1), (b'150: 1', 1), (b'151: 1', 1), (b'152: 1', 1), (b'153: 1', 1), (b'154: 1', 1), (b'155: 1', 1), (b'156: 1', 1), (b'157: 1', 1), (b'158: 1', 1), (b'159: 1', 1), (b'160: 1', 1), (b'161: 1', 1), (b'162: 1', 1), (b'163: 1', 1), (b'164: 1', 1), (b'165: 1', 1), (b'166: 1', 1), (b'167: 1', 1), (b'168: 1', 1), (b'169: 1', 1), (b'170: 1', 1), (b'171: 1', 1), (b'172: 1', 1), (b'173: 1', 1), (b'174: 1', 1), (b'175: 1', 1), (b'176: 1', 1), (b'177: 1', 1), (b'178: 1', 1), (b'179: 1', 1), (b'180: 1', 1), (b'181: 1', 1), (b'182: 1', 1), (b'183: 1', 1), (b'184: 1', 1), (b'185: 1', 1), (b'186: 1', 1), (b'187: 1', 1), (b'188: 1', 1), (b'189: 1', 1), (b'190: 1', 1), (b'191: 1', 1), (b'192: 1', 1), (b'193: 1', 1), (b'194: 1', 1), (b'195: 1', 1), (b'196: 1', 1), (b'197: 1', 1), (b'198: 1', 1), (b'199: 1', 1), (b'200: 1', 1), (b'201: 1', 1), (b'202: 1', 1), (b'203: 1', 1), (b'204: 1', 1), (b'205: 1', 1), (b'206: 1', 1), (b'207: 1', 1), (b'208: 1', 1), (b'209: 1', 1), (b'210: 1', 1), (b'211: 1', 1), (b'212: 1', 1), (b'213: 1', 1), (b'214: 1', 1), (b'215: 1', 1), (b'216: 1', 1), (b'217: 1', 1), (b'218: 1', 1), (b'219: 1', 1), (b'220: 1', 1), (b'221: 1', 1), (b'222: 1', 1), (b'223: 1', 1), (b'224: 1', 1), (b'225: 1', 1), (b'226: 1', 1), (b'227: 1', 1), (b'228: 1', 1), (b'229: 1', 1), (b'230: 1', 1), (b'231: 1', 1), (b'232: 1', 1), (b'233: 1', 1), (b'234: 1', 1), (b'235: 1', 1), (b'236: 1', 1), (b'237: 1', 1), (b'238: 1', 1), (b'239: 1', 1), (b'240: 1', 1), (b'241: 1', 1), (b'242: 1', 1), (b'243: 1', 1), (b'244: 1', 1), (b'245: 1', 1), (b'246: 1', 1), (b'247: 1', 1), (b'248: 1', 1), (b'249: 1', 1), (b'250: 1', 1), (b'251: 1', 1), (b'252: 1', 1), (b'253: 1', 1), (b'254: 1', 1), (b'255: 1', 1), (b'256: 1', 1), (b'257: 1', 1), (b'258: 1', 1), (b'259: 1', 1), (b'260: 1', 1), (b'261: 1', 1), (b'262: 1', 1), (b'263: 1', 1), (b'264: 1', 1), (b'265: 1', 1), (b'266: 1', 1), (b'267: 1', 1), (b'268: 1', 1), (b'269: 1', 1), (b'270: 1', 1), (b'271: 1', 1), (b'272: 1', 1), (b'273: 1', 1), (b'274: 1', 1), (b'275: 1', 1), (b'276: 1', 1), (b'277: 1', 1), (b'278: 1', 1), (b'279: 1', 1), (b'280: 1', 1), (b'281: 1', 1), (b'282: 1', 1), (b'283: 1', 1), (b'284: 1', 1), (b'285: 1', 1), (b'286: 1', 1), (b'287: 1', 1), (b'288: 1', 1), (b'289: 1', 1), (b'290: 1', 1), (b'291: 1', 1), (b'292: 1', 1), (b'293: 1', 1), (b'294: 1', 1), (b'295: 1', 1), (b'296: 1', 1), (b'297: 1', 1), (b'298: 1', 1), (b'299: 1', 1), (b'300: 1', 1), (b'301: 1', 1), (b'302: 1', 1), (b'303: 1', 1), (b'304: 1', 1), (b'305: 1', 1), (b'306: 1', 1), (b'307: 1', 1), (b'308: 1', 1), (b'309: 1', 1), (b'310: 1', 1), (b'311: 1', 1), (b'312: 1', 1), (b'313: 1', 1), (b'314: 1', 1), (b'315: 1', 1), (b'316: 1', 1), (b'317: 1', 1), (b'318: 1', 1), (b'319: 1', 1), (b'320: 1', 1), (b'321: 1', 1), (b'322: 1', 1), (b'323: 1', 1), (b'324: 1', 1), (b'325: 1', 1), (b'326: 1', 1), (b'327: 1', 1), (b'328: 1', 1), (b'329: 1', 1), (b'330: 1', 1), (b'331: 1', 1), (b'332: 1', 1), (b'333: 1', 1), (b'334: 1', 1), (b'335: 1', 1), (b'336: 1', 1), (b'337: 1', 1), (b'338: 1', 1), (b'339: 1', 1), (b'340: 1', 1), (b'341: 1', 1), (b'342: 1', 1), (b'343: 1', 1), (b'344: 1', 1), (b'345: 1', 1), (b'346: 1', 1), (b'347: 1', 1), (b'348: 1', 1), (b'349: 1', 1), (b'350: 1', 1), (b'351: 1', 1), (b'352: 1', 1), (b'353: 1', 1), (b'354: 1', 1), (b'355: 1', 1), (b'356: 1', 1), (b'357: 1', 1), (b'358: 1', 1), (b'359: 1', 1), (b'360: 1', 1), (b'361: 1', 1), (b'362: 1', 1), (b'363: 1', 1), (b'364: 1', 1), (b'365: 1', 1), (b'366: 1', 1), (b'367: 1', 1), (b'368: 1', 1), (b'369: 1', 1), (b'370: 1', 1), (b'371: 1', 1), (b'372: 1', 1), (b'373: 1', 1), (b'374: 1', 1), (b'375: 1', 1), (b'376: 1', 1), (b'377: 1', 1), (b'378: 1', 1), (b'379: 1', 1), (b'380: 1', 1), (b'381: 1', 1), (b'382: 1', 1), (b'383: 1', 1), (b'384: 1', 1), (b'385: 1', 1), (b'386: 1', 1), (b'387: 1', 1), (b'388: 1', 1), (b'389: 1', 1), (b'390: 1', 1), (b'391: 1', 1), (b'392: 1', 1), (b'393: 1', 1), (b'394: 1', 1), (b'395: 1', 1), (b'396: 1', 1), (b'397: 1', 1), (b'398: 1', 1), (b'399: 1', 1), (b'400: 1', 1), (b'401: 1', 1), (b'402: 1', 1), (b'403: 1', 1), (b'404: 1', 1), (b'405: 1', 1), (b'406: 1', 1), (b'407: 1', 1), (b'408: 1', 1), (b'409: 1', 1), (b'410: 1', 1), (b'411: 1', 1), (b'412: 1', 1), (b'413: 1', 1), (b'414: 1', 1), (b'415: 1', 1), (b'416: 1', 1), (b'417: 1', 1), (b'418: 1', 1), (b'419: 1', 1), (b'420: 1', 1), (b'421: 1', 1), (b'422: 1', 1), (b'423: 1', 1), (b'424: 1', 1), (b'425: 1', 1), (b'426: 1', 1), (b'427: 1', 1), (b'428: 1', 1), (b'429: 1', 1), (b'430: 1', 1), (b'431: 1', 1), (b'432: 1', 1), (b'433: 1', 1), (b'434: 1', 1), (b'435: 1', 1), (b'436: 1', 1), (b'437: 1', 1), (b'438: 1', 1), (b'439: 1', 1), (b'440: 1', 1), (b'441: 1', 1), (b'442: 1', 1), (b'443: 1', 1), (b'444: 1', 1), (b'445: 1', 1), (b'446: 1', 1), (b'447: 1', 1), (b'448: 1', 1), (b'449: 1', 1), (b'450: 1', 1), (b'451: 1', 1), (b'452: 1', 1), (b'453: 1', 1), (b'454: 1', 1), (b'455: 1', 1), (b'456: 1', 1), (b'457: 1', 1), (b'458: 1', 1), (b'459: 1', 1), (b'460: 1', 1), (b'461: 1', 1), (b'462: 1', 1), (b'463: 1', 1), (b'464: 1', 1), (b'465: 1', 1), (b'466: 1', 1), (b'467: 1', 1), (b'468: 1', 1), (b'469: 1', 1), (b'470: 1', 1), (b'471: 1', 1), (b'472: 1', 1), (b'473: 1', 1), (b'474: 1', 1), (b'475: 1', 1), (b'476: 1', 1), (b'477: 1', 1), (b'478: 1', 1), (b'479: 1', 1), (b'480: 1', 1), (b'481: 1', 1), (b'482: 1', 1), (b'483: 1', 1), (b'484: 1', 1), (b'485: 1', 1), (b'486: 1', 1), (b'487: 1', 1), (b'488: 1', 1), (b'489: 1', 1), (b'490: 1', 1), (b'491: 1', 1), (b'492: 1', 1), (b'493: 1', 1), (b'494: 1', 1), (b'495: 1', 1), (b'496: 1', 1), (b'497: 1', 1), (b'498: 1', 1), (b'499: 1', 1)])
E Unexpected: dict_items([])
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20']
WARNING apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:250 Timing out on waiting for job 2021-08-09_05_38_37-4716910266292310759 after 364 seconds
ERROR apache_beam.io.gcp.tests.pubsub_matcher:pubsub_matcher.py:162 Timeout after 400 sec. Received 0 messages from projects/apache-beam-testing/subscriptions/wc_subscription_output38146f40-13de-46fc-be41-e0b30d4e6cdf.
- generated xml file: <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/pytest_preCommitIT-df-py36.xml> -
========================== 1 failed in 957.46 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2 FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 81
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:preCommitIT_streaming_V2'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 42s
106 actionable tasks: 78 executed, 28 from cache
Publishing build scan...
https://gradle.com/s/jxvop3odjjgpy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org