You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/10 18:41:32 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8231
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8231/display/redirect?page=changes>
Changes:
[noreply] Added type annotations to some combiners missing it. (#15414)
[noreply] [BEAM-12634] JmsIO auto scaling feature (#15464)
[noreply] [BEAM-12662] Get Flink version from cluster. (#15223)
------------------------------------------
[...truncated 123.67 KB...]
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
> result = self.run_pipeline(experiment='beam_fn_api')
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:43: in run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py:176: in apply_and_run
result = pipeline.run()
apache_beam/pipeline.py:542: in run
self._options).run(False)
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210910182558506794-5768'
createTime: '2021-09-10T18:25:59.949896Z'
...021-09-10T18:25:59.949896Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7efeefa46cf8>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml> -
==================== 4 failed, 2 warnings in 791.64 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-181217005449927
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:68e510d62bc7e21adc241c4bf5e7811d4ab35f681d2f542e21c574f901367bde
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
[gw0] [ 50%] FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:a6a32375-76cb-44ac-886f-b368ecddaed2
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:d8e01f4f892910be1185a57abd9fb667a75a278262e3748ab1cb5ea982896b51
Deleted: sha256:ed2431b7ce8bc4651b33def5b49bf637314b6c73b2ac904b2dae988bfd807cb1
Deleted: sha256:ad2553f69ef8a3fce5d880da7c248867cfe203d26d5b72c9c362127e52f53e4e
Deleted: sha256:5ad64f455985fff81f86af27951679794a7ff8e08ba481301fedc0c20615a8fc
Deleted: sha256:18e0ba12141ca62cf8f6e63bd28ed3830af79efc0682dae4b9357ac329a2af01
Deleted: sha256:277d788677def706087298cf73f07407816d0c6d9e96ffae7eb8e646f4d5a826
Deleted: sha256:0fde726c6313ed355318f08b3a4737b226f5fccdfe23d93b00150defe24ac566
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:68e510d62bc7e21adc241c4bf5e7811d4ab35f681d2f542e21c574f901367bde
Associated tags:
- 20210910-181217005449927
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-181217005449927
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-181217005449927].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:68e510d62bc7e21adc241c4bf5e7811d4ab35f681d2f542e21c574f901367bde].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:d8e01f4f892910be1185a57abd9fb667a75a278262e3748ab1cb5ea982896b51
Associated tags:
- a6a32375-76cb-44ac-886f-b368ecddaed2
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:a6a32375-76cb-44ac-886f-b368ecddaed2].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:d8e01f4f892910be1185a57abd9fb667a75a278262e3748ab1cb5ea982896b51].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:adcd186acc19586d0948dfda7db8548998fcb82001c18703f955c33888fc16e7
Associated tags:
- 3ee4b483-cb80-4852-ac85-4bb896266e89
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:3ee4b483-cb80-4852-ac85-4bb896266e89].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:adcd186acc19586d0948dfda7db8548998fcb82001c18703f955c33888fc16e7].
Removed the container
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:4b2ff9afbc79b123c7caf6552dc80d6f1afe744cb5451c4fb618ed17ac1cf3a9
Associated tags:
- 20210910-181216996579364
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210910-181216996579364
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210910-181216996579364].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:4b2ff9afbc79b123c7caf6552dc80d6f1afe744cb5451c4fb618ed17ac1cf3a9].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:dc9ead970e1982183d6aa5fca0b58b1439c01517aaa77658819c0b684cfb21a8
Associated tags:
- b76958f5-bd0d-4426-84f5-a4364c6ade57
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:b76958f5-bd0d-4426-84f5-a4364c6ade57].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:dc9ead970e1982183d6aa5fca0b58b1439c01517aaa77658819c0b684cfb21a8].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:60ed9a1612f7a2d9505150ac8daf0f29a1dff74c9e059ddeedcd17a8ee5d271e
Associated tags:
- cd0be990-f07f-4810-8b4a-9a31679862f7
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:cd0be990-f07f-4810-8b4a-9a31679862f7].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:60ed9a1612f7a2d9505150ac8daf0f29a1dff74c9e059ddeedcd17a8ee5d271e].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:48be161bbb14d1703b3dc2b83e6754b8bde69ca099e59200dc39b93fe0ee53c9
Associated tags:
- 20210910-181223269403059
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210910-181223269403059
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210910-181223269403059].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:48be161bbb14d1703b3dc2b83e6754b8bde69ca099e59200dc39b93fe0ee53c9].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3b03d57a6596920b80cd3fb949e9368304167a53aa8aaa1b60e083f0006f3a9d
Associated tags:
- 086e399e-5ec2-459b-9dee-468d8bf03646
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:086e399e-5ec2-459b-9dee-468d8bf03646].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3b03d57a6596920b80cd3fb949e9368304167a53aa8aaa1b60e083f0006f3a9d].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c2bcc129720a5ca06cd66620b8c6d0fa9d69e1e159e8b957cca67d8c76118d99
Associated tags:
- 61831813-3965-4121-9ce8-8a3910a197de
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:61831813-3965-4121-9ce8-8a3910a197de].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c2bcc129720a5ca06cd66620b8c6d0fa9d69e1e159e8b957cca67d8c76118d99].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 41m 1s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/y3x3wk5jt3i5i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #8244
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8244/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8243
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8243/display/redirect?page=changes>
Changes:
[rohde.samuel] [BEAM-12842] Add timestamp to test work item to deflake
[suztomo] [BEAM-12873] HL7v2IO: to leave schematizedData null, not empty
------------------------------------------
[...truncated 147.77 KB...]
prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:64:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:108: in _run_wordcount_it
run_wordcount(
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210913182131150071-7102'
createTime: '2021-09-13T18:21:32.680208Z'
...021-09-13T18:21:32.680208Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f05755b7640>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
> raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1632: DataflowRuntimeException
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self):
> self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:108: in _run_wordcount_it
run_wordcount(
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f0575554940>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f057593bbe0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f0575978580>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
result = self.run_pipeline(experiment='beam_fn_api')
errors = metric_result_matchers.verify_all(
result.metrics().all_metrics(),
dataflow_exercise_metrics_pipeline.metric_matchers())
> self.assertFalse(errors, str(errors))
E AssertionError: "Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>\nActual MetricResults:\n" is not false : Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>
E Actual MetricResults:
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:60: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
==================== 4 failed, 3 warnings in 551.55 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210913-181233714496878
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:76918bef6dd4a69e889b9e805a0da11b35b4a38b85a0f255a576355f98bb7224
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:49f1325a-effc-4de6-bfa4-8e756f3cbb67
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:47d57d74ca77e43e38c46df84e3b12ffed6b564fa8e18e23ea884e1eefb74e0a
Deleted: sha256:43ca3c8473f6f21fe6ca1fba8f435b7b1e845a73f773667e57730c9d7b4587b1
Deleted: sha256:a7629d16ef8920970aa9586de881d60eb2db126b3372b3fce187d6ece717c7d3
Deleted: sha256:9d7a0e6bad789ecf37def8cffff3fe8ec3117567d5d94ce01f14442faa197b45
Deleted: sha256:e28ed087d5aaef6a680e50cc43ffc6cffd2d779016951a4f0d8cfb21ec5bc517
Deleted: sha256:44da2ba5e98dc62a96d769f1fdb789b6663f4bd3941e5ee49e63eb0bcefb72ec
Deleted: sha256:c3d283891d647e7c20163619edf9f22c188733f4c7032e4bf40293d652a74ce2
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:76918bef6dd4a69e889b9e805a0da11b35b4a38b85a0f255a576355f98bb7224
Associated tags:
- 20210913-181233714496878
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210913-181233714496878
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210913-181233714496878].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:76918bef6dd4a69e889b9e805a0da11b35b4a38b85a0f255a576355f98bb7224].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:47d57d74ca77e43e38c46df84e3b12ffed6b564fa8e18e23ea884e1eefb74e0a
Associated tags:
- 49f1325a-effc-4de6-bfa4-8e756f3cbb67
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:49f1325a-effc-4de6-bfa4-8e756f3cbb67].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:47d57d74ca77e43e38c46df84e3b12ffed6b564fa8e18e23ea884e1eefb74e0a].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:66a8357a93cb06625d940d9d73a7aa9825426c97a07eb42c5bcf1da8ef6c7fcc
Associated tags:
- 44c5bfa6-97f1-49c4-aafd-a622946b910e
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:44c5bfa6-97f1-49c4-aafd-a622946b910e].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:66a8357a93cb06625d940d9d73a7aa9825426c97a07eb42c5bcf1da8ef6c7fcc].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 23m 56s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/gqfemqv75v5b4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8242
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8242/display/redirect>
Changes:
------------------------------------------
[...truncated 1.19 MB...]
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=900 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)")
do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=900 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)")
do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=900 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:8dc753f59929dca64e51ff058377ae2c70fa6b8f5cf47f8fa7868cf6ddcdc1ec
Associated tags:
- 20210913-130226772704993
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210913-130226772704993
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210913-130226772704993].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:8dc753f59929dca64e51ff058377ae2c70fa6b8f5cf47f8fa7868cf6ddcdc1ec].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9e66086fbf9a919035976830f700b3b7d5e9fdb0d711efb00f38a40efccd29d4
Associated tags:
- b69a10f5-90cd-470c-ba5c-3e95b2fd1df0
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:b69a10f5-90cd-470c-ba5c-3e95b2fd1df0].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9e66086fbf9a919035976830f700b3b7d5e9fdb0d711efb00f38a40efccd29d4].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:0f26e9f3b8d6af598e7ba80f208e9d13fece7c6cedd7b7e0c99504efae34f0d5
Associated tags:
- b71e7e8f-8c79-4b66-b01f-d933f68034b7
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:b71e7e8f-8c79-4b66-b01f-d933f68034b7].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:0f26e9f3b8d6af598e7ba80f208e9d13fece7c6cedd7b7e0c99504efae34f0d5].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c87fa96b4b5c6d6f7c29651c583c73a49f75b028743482c7f16bf0deb4aec86d
Associated tags:
- 20210913-130226780474230
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210913-130226780474230
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210913-130226780474230].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:6cb688769884bb934c45581ec0c1b5a20f68afa4b3b85b3c775e2da670519670
Associated tags:
- 20210913-130226738396298
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210913-130226738396298
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210913-130226738396298].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:6cb688769884bb934c45581ec0c1b5a20f68afa4b3b85b3c775e2da670519670].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:a1f32a5afe4a666f843e1a818fd6924a17e8b55ada27f03850ee4f82ea7cafc4
Associated tags:
- df5beab5-22c3-46bc-85fd-a6b24294e41e
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:df5beab5-22c3-46bc-85fd-a6b24294e41e].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:a1f32a5afe4a666f843e1a818fd6924a17e8b55ada27f03850ee4f82ea7cafc4].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c87fa96b4b5c6d6f7c29651c583c73a49f75b028743482c7f16bf0deb4aec86d].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:55692ec541c761881b607bb377b6900ef3b4c434521718692994ef6baf07e167
Associated tags:
- 0acdb69d-e149-4906-8edf-b20ce119918c
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:0acdb69d-e149-4906-8edf-b20ce119918c].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:55692ec541c761881b607bb377b6900ef3b4c434521718692994ef6baf07e167].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:e7d9d988a6fb44abc634e4c3cb4a8d53f17986f1aa9bea44013007ee4c6e5d66
Associated tags:
- 13efbc7a-6a78-406f-b83b-9b267eaf31cd
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:13efbc7a-6a78-406f-b83b-9b267eaf31cd].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:e7d9d988a6fb44abc634e4c3cb4a8d53f17986f1aa9bea44013007ee4c6e5d66].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:8595a50a34cb0e969debcdba583acd21310efa657cc21f251d73c9a150c7b3e8
Associated tags:
- 1cfaf544-4fb4-48b5-a2e7-ce80b5d14504
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:1cfaf544-4fb4-48b5-a2e7-ce80b5d14504].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:8595a50a34cb0e969debcdba583acd21310efa657cc21f251d73c9a150c7b3e8].
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 16m 18s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/szjvnxehltpje
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8241
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8241/display/redirect>
Changes:
------------------------------------------
[...truncated 144.71 KB...]
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f7deb4136d8>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f7deae4fe10>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f7deb05bac8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:542: in run
self._options).run(False)
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f7debc16048>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f7dea5aba90>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f7dea5da4e0>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
result = self.run_pipeline(experiment='beam_fn_api')
errors = metric_result_matchers.verify_all(
result.metrics().all_metrics(),
dataflow_exercise_metrics_pipeline.metric_matchers())
> self.assertFalse(errors, str(errors))
E AssertionError: "Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>\nActual MetricResults:\n" is not false : Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>
E Actual MetricResults:
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:60: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml> -
==================== 4 failed, 2 warnings in 519.58 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210913-062141120121269
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:7c0c903a3d4f02ccf9b0649715d742bfab6f38790481e64980b5961fbf7ccbef
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:b437188b-27f2-4daf-8563-6b23a3f79886
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:32c57a34c213ea133941a482c5ddace6feaef08a1fbbc8bc39efb36c08066b88
Deleted: sha256:7473b61027c2c8eace555194b983a28300d1441fbc53a2a23687bdfa9e56d835
Deleted: sha256:ca8f05b069f00f8e57015d7cacc0507f72ab5fde19d3d8c41dbd1bc696e8b762
Deleted: sha256:e867f045b8f29abe3b74c5198a359d8fe5f7bd45aa0a736d29122aba25f887b4
Deleted: sha256:b8bd6389e97c7731a453c432410a41a313be20aa5331157762056718e8b8bb06
Deleted: sha256:35453bb4fd58c4194a5e20a3090d6acd48eb7a295aae8cbb6d97c1904a7791e4
Deleted: sha256:3e1359ab26132b95ee3c9b4c2281dd1defe072ba65ab2800a4f154ac996a4482
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:7c0c903a3d4f02ccf9b0649715d742bfab6f38790481e64980b5961fbf7ccbef
Associated tags:
- 20210913-062141120121269
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210913-062141120121269
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210913-062141120121269].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:7c0c903a3d4f02ccf9b0649715d742bfab6f38790481e64980b5961fbf7ccbef].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:32c57a34c213ea133941a482c5ddace6feaef08a1fbbc8bc39efb36c08066b88
Associated tags:
- b437188b-27f2-4daf-8563-6b23a3f79886
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:b437188b-27f2-4daf-8563-6b23a3f79886].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:32c57a34c213ea133941a482c5ddace6feaef08a1fbbc8bc39efb36c08066b88].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:44c5cb730385aab7aa64ec250b4bf70e2ca9d3ee050da07298fddd4874632428
Associated tags:
- 36fa042b-1645-45a5-affd-de0e8747335f
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:36fa042b-1645-45a5-affd-de0e8747335f].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:44c5cb730385aab7aa64ec250b4bf70e2ca9d3ee050da07298fddd4874632428].
Removed the container
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:73d8f2b9c7200b36e32c9fba4860940ca6beac5763f24c1e446322ab09fee0e9
Associated tags:
- 20210913-062141365740794
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210913-062141365740794
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210913-062141365740794].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:73d8f2b9c7200b36e32c9fba4860940ca6beac5763f24c1e446322ab09fee0e9].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:97dd765585e820a7c41ea82babad635b997e2108f38c9ddfdf0687fdb3ae4729
Associated tags:
- 974b3359-6021-47ec-be78-b0a2ce9191db
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:974b3359-6021-47ec-be78-b0a2ce9191db].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:97dd765585e820a7c41ea82babad635b997e2108f38c9ddfdf0687fdb3ae4729].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:05100c5cfd8e1973863d2851f306eea83312e938b4e84a207385839d685f8af8
Associated tags:
- 330fbb27-b610-45eb-9741-6a4d15fe3fe0
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:330fbb27-b610-45eb-9741-6a4d15fe3fe0].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:05100c5cfd8e1973863d2851f306eea83312e938b4e84a207385839d685f8af8].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 32m 53s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jgg5ouxjrupl6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8240
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8240/display/redirect>
Changes:
------------------------------------------
[...truncated 115.14 KB...]
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:542: in run
self._options).run(False)
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f2527cd4be0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f2527332f60>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f2527394278>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
result = self.run_pipeline(experiment='beam_fn_api')
errors = metric_result_matchers.verify_all(
result.metrics().all_metrics(),
dataflow_exercise_metrics_pipeline.metric_matchers())
> self.assertFalse(errors, str(errors))
E AssertionError: "Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>\nActual MetricResults:\n" is not false : Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>
E Actual MetricResults:
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:60: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml> -
==================== 4 failed, 2 warnings in 428.48 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210913-001507405711375
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:1eeb91370281d35f08af02c0c37f7dcc3e5e3dabaa69091ee0e0fe7c61e0b1b1
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:fbfe9829-3aef-4824-a6cc-d445156f06c2
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:4666f83414577e041e395c8e151cf0b83ce8d9802f7df175191380f86c0d537f
Deleted: sha256:3d1987926537345418866a96f4ec4547688277fbfd2074f5d4e553ec0605494a
Deleted: sha256:b253156e9f54e47d5134950f9cbcf1f470173ee1ec6748de3e4ea5b2194708c2
Deleted: sha256:04abc5f8d877c46b2fee4033bae7d82e6e34e16d3fa35c06f8b931685da108a0
Deleted: sha256:dfd8cb6dc65869d1d94e772c8c14b60d819b3d33130735d363b2266c0f7a49c3
Deleted: sha256:df621e14949cc20f02d4af1ed1ff9e480efb86b15c79db1081fa3b0d3ceb8eff
Deleted: sha256:7265ea9583178c23f741a2d5bdac7acc939504e4ed5788d4f330da304c83abc3
Deleted: sha256:5b1503e8238be9419a33ed80c8f9eb69dccdfab8f134fb112110672fdde52ba4
Deleted: sha256:8a4c4037d1053916354da497fed94214c1301fab22e36c2293227ff3ec97cdeb
Deleted: sha256:6be9060ba00a2221e2dba5575d2ce24cbf7f1dc1be68431c1d3324db20a501d8
Deleted: sha256:db16328d1c61fa6217f223ed93511c819f324e7fd436e5fc78cca5714058696b
Deleted: sha256:172ec1966ff8ebb8f11ad72c2ffd59008422da243fa68002f349d9483d862277
Deleted: sha256:b15c66560be9e79f2f9ca44671b457af8c446289d63a1556527b09ca1a2ef966
Deleted: sha256:ecc2bc8576fb7093697783948b611971505d53621441b09a8035c1792192c23b
Deleted: sha256:ac6d123ea9fe7aa7f9b1c806ab092106cad0bd3b14b28e2de30f6ce6b0e2eb20
Deleted: sha256:a226660597bcd1b8467f046c90dfab9be61cfc0d795ab46657f09c5b38e0cc59
Deleted: sha256:2e350544ca1614a17a04196861d1129db02e99102908f2751edc933c20b29437
Deleted: sha256:40f6164e7d50485b8d49aeaf134adb0ce9b564b9e61792552ecd20865fe335fd
Deleted: sha256:e62cb99dc9d2b7d2c4cc35c326f10aff5ae0ebd1a5322bc9e9470ce295e9c59b
Deleted: sha256:7df3d7371c6161f21f2eb96fa49186dc5d142e53fe733be7c17b81c1050db489
Deleted: sha256:8abdf99580162c37e81dd65d7266784180d1e2ba5f00b34c31b6c52ddd2910e8
Deleted: sha256:3cc5b71d1f7838891747145cc3b2a2be521d2bd30b0d512bbb82f15f097df79b
Deleted: sha256:545e6cfc407aefee69c40996b270f36bce88d172380112302cf6c8e085a55d47
Deleted: sha256:e51d221317dba4dff0ad62b418894baf40d7d180ff6357d2ed228eb80e40a89e
Deleted: sha256:eb71030da392ca3ae600c6660840c2fbe2ec20f8c6f790fc292b99df26679b23
Deleted: sha256:a6aadaa78909c983782643d73bef3a4c8a593b5dc48ccaf6dbf2c8c1df819759
Deleted: sha256:e2eceaabef959d38f1e5da063a3f68927f5d45a30e2f8f4117237205e28703d9
Deleted: sha256:71aaa3d3b7a5303d0b9634940a878c1db04cb501469f41c5f0da8431e697f476
Deleted: sha256:6d546cf6830a0c5ec837d74de4633709ca4600e1bc18402ecb04bca4775db2f9
Deleted: sha256:13f0ef2345509adce8cf9db0aec9296033d320fcec08407eb5e1fdc82c7d7ce6
Deleted: sha256:57d6bf396c916b4ff5e835e3f20c1514b9667c661d99eb3f6861d3a01ac05724
Deleted: sha256:8f503e1422707a4375e4a9673b390c5b58644fc30c57b9deed1d7fb0f6325e9a
Deleted: sha256:8ad05b49d4fc9139097e0bbf6a8d5907f9c80350c7aaeccd7ab85a9e44e9ce98
Deleted: sha256:d71bf28413408b7ab7c8953ae3c9cf35712843747f8d149826da536717087848
Deleted: sha256:95db30cff33d50483d30baef9ecff0ff7aeed2cf31fcbbe5f469feb2971041cb
Deleted: sha256:68ddd1312f5d0444020cadd2177fc7dbe106dc85912d801822e97dd7a2a1533f
Deleted: sha256:d9335c2ba1b468e6579f0fa2c677d0b574064fa0914881fa0ac5326b8fd74992
Deleted: sha256:bf1b523562711bc7072ec67f80fdec839031a76be6ec902c4ea7d830036bfb39
Deleted: sha256:df7e171904e8f5fc3afaa7a10662e9fc1fc5c846f948ab73a1e913e28983d061
Deleted: sha256:319cc9752a61bd7d36d1e310ae0fd801447f5944f7b9c7a3510ce99093759b10
Deleted: sha256:b2a64c6c6c5771c6f7a229eeaf6a675e7e233c6cf224724f1991ab6c2a4bacaf
Deleted: sha256:c39255c904bc11828e2ddfce1cf49e79bafa69dc91c8372d4fba67485ab5c708
Deleted: sha256:76d9fdf6bdc4f09b274cf209814ff25a93bd825e0eefdf52c7f61a82d9a6da6f
Deleted: sha256:93f171cc5bad227e65abea5bb9b4273aad92551fc2e4d0e00782bcf6999ebbe5
Deleted: sha256:16f6f6ae0f590d9fcf25219740700383ac8382d018769995792709a7f22b8ffe
Deleted: sha256:696a55701e4597643d190e7416b2eeb670bea024207cd38ae621468f400c770d
Deleted: sha256:b298804b03920dcf04b582cb4c807b8edbb06f0ea99333849c56427d45d287cd
Deleted: sha256:ef72b4ae62117cb2201eef1caa52c77b64884e10f6da181f6a0d13a1ae7572e3
Deleted: sha256:eb3ce820418325bd8302921712e380c6e8131d5ac47fb18c4d8e004e27bc9ff5
Deleted: sha256:4cb8d5c9589f825f07b7c900ab52e811e64fa6443f5d0132e2a098100ec0bba4
Deleted: sha256:fbecdc3808c06ad4b4c6ae4a53b729ad345a986a132cd5c897ff653a98105840
Deleted: sha256:972e0beb4e4a62608ea3b016db69ce2e300a49436f1c746c35a6226a26411908
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:1eeb91370281d35f08af02c0c37f7dcc3e5e3dabaa69091ee0e0fe7c61e0b1b1
Associated tags:
- 20210913-001507405711375
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210913-001507405711375
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210913-001507405711375].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:1eeb91370281d35f08af02c0c37f7dcc3e5e3dabaa69091ee0e0fe7c61e0b1b1].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:4666f83414577e041e395c8e151cf0b83ce8d9802f7df175191380f86c0d537f
Associated tags:
- fbfe9829-3aef-4824-a6cc-d445156f06c2
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:fbfe9829-3aef-4824-a6cc-d445156f06c2].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:4666f83414577e041e395c8e151cf0b83ce8d9802f7df175191380f86c0d537f].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:6fb5eeb9a418a06df525ffb0f6e56489454030f37f65256ea0b6b25143e01082
Associated tags:
- e28a1dc4-1014-4d76-a109-7a888cbc482c
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:e28a1dc4-1014-4d76-a109-7a888cbc482c].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:6fb5eeb9a418a06df525ffb0f6e56489454030f37f65256ea0b6b25143e01082].
Removed the container
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:abf245eb263f832e303751d1f1d515c13e6de7e8bd2dc9b36b2f5381f1c1784a
Associated tags:
- 20210913-001515519949857
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210913-001515519949857
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210913-001515519949857].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:abf245eb263f832e303751d1f1d515c13e6de7e8bd2dc9b36b2f5381f1c1784a].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:288e668e5e00a5722ce75f2da7977844091e3b025036fdaef49d3e705ba8a4e5
Associated tags:
- 55daab3f-29e8-498e-afc0-529f27864524
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:55daab3f-29e8-498e-afc0-529f27864524].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:288e668e5e00a5722ce75f2da7977844091e3b025036fdaef49d3e705ba8a4e5].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:92520be59e4f4806cd213a00e2cef6557b996e73bbb4479e3e335f5fd7a1b1e3
Associated tags:
- 558f0eb7-7c2b-4751-b26c-cc434629d271
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:558f0eb7-7c2b-4751-b26c-cc434629d271].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:92520be59e4f4806cd213a00e2cef6557b996e73bbb4479e3e335f5fd7a1b1e3].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 25m 41s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/pekigz2n6d5mg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8239
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8239/display/redirect>
Changes:
------------------------------------------
[...truncated 115.33 KB...]
def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:67:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:542: in run
self._options).run(False)
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210912182425560538-7102'
createTime: '2021-09-12T18:24:26.898828Z'
...021-09-12T18:24:26.898828Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f67ca843f60>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:542: in run
self._options).run(False)
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f67cacff6a0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f67ca789ac8>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f67c9e81710>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
result = self.run_pipeline(experiment='beam_fn_api')
errors = metric_result_matchers.verify_all(
result.metrics().all_metrics(),
dataflow_exercise_metrics_pipeline.metric_matchers())
> self.assertFalse(errors, str(errors))
E AssertionError: "Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>\nActual MetricResults:\n" is not false : Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>
E Actual MetricResults:
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:60: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml> -
==================== 4 failed, 2 warnings in 644.59 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210912-181412168536995
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:a506ca3f9451234745cccce642220f44f3b3c98b5ae2298f76d23ec4e3b45096
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:668269bf-7016-40eb-a70f-1ae5ee79d5d1
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:e9a0fc0c378b3567f51fab0f6d5f490088a13825254882906d22d439201993da
Deleted: sha256:9d846adae497f0711243a6eb4e14f36cf4a414c297737d5866532ddd1f5c944b
Deleted: sha256:32d3541d4bd8ae94ccf95b17714595697b88f18fac2798f9b54b3e8ec0cbc572
Deleted: sha256:799f09d7d3e36eee53bc520c9789da7dfad7309e19c63f1382c3281e197701a0
Deleted: sha256:ed425699f3826d31f4c44a2650b14053adcf928c7f9c739b37f34e4c8bb4960f
Deleted: sha256:a3ce379bdd1c9ed95b1eac5da26087aa07de513bf7697e423cb2bee3878daf50
Deleted: sha256:f9be84da87c6f61239be847a816dedfa17f73300132609719eef19f79aa77db8
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:a506ca3f9451234745cccce642220f44f3b3c98b5ae2298f76d23ec4e3b45096
Associated tags:
- 20210912-181412168536995
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210912-181412168536995
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210912-181412168536995].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:a506ca3f9451234745cccce642220f44f3b3c98b5ae2298f76d23ec4e3b45096].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:e9a0fc0c378b3567f51fab0f6d5f490088a13825254882906d22d439201993da
Associated tags:
- 668269bf-7016-40eb-a70f-1ae5ee79d5d1
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:668269bf-7016-40eb-a70f-1ae5ee79d5d1].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:e9a0fc0c378b3567f51fab0f6d5f490088a13825254882906d22d439201993da].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:79a43c0a19df831dae4ef166e2bb073ca23cfc60000d2cf93cbafde2b68abadb
Associated tags:
- b3a1a080-f9ed-4417-b377-c866dba2dafb
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:b3a1a080-f9ed-4417-b377-c866dba2dafb].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:79a43c0a19df831dae4ef166e2bb073ca23cfc60000d2cf93cbafde2b68abadb].
Removed the container
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 48s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ho7r42fq2jykw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8238
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8238/display/redirect>
Changes:
------------------------------------------
[...truncated 55.76 KB...]
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
> result = self.run_pipeline(experiment='beam_fn_api')
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:43: in run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py:176: in apply_and_run
result = pipeline.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210912124307130553-3474'
createTime: '2021-09-12T12:43:08.400379Z'
...021-09-12T12:43:08.400379Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f3ded9c8310>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
> raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1632: DataflowRuntimeException
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
============== 2 failed, 2 passed, 3 warnings in 1318.65 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-121903222604151
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:0840de1543ecd50b5e5a241b773c3697fa3fbea661d062ec59308946f2b754e2
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:86b3af13-f7da-44f3-8dd6-5f22e6549ca0
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:2ec2e41063f5fc332cd992f10dd2d82e49a1931e030a7ea8b4c6cceb6b883fa7
Deleted: sha256:89d945d0dd3dbda91e2f61bc206d16a8de4e153a13034a4be991ccab0d82d09e
Deleted: sha256:5fff66ee5a03e5f1c3e284ea318ad43b56e02c5b5a81e39c40801c0950698479
Deleted: sha256:7a468a7df57b578cf1ca56d23f457dcf2554cceaeb24870c0d2d078cc24fc5ea
Deleted: sha256:b996c58de296458376864164fe684839f5963bf8de79ee7c796e08b7cec89379
Deleted: sha256:9fad7ee84fdd186dac5e54d4f6eccca30544428fb08ee7a6f3987db5aede871a
Deleted: sha256:fb470a42f74eba0f56fcd190795504a66e898db2d275e4547487e93368ccacda
Deleted: sha256:c7fc813c6c3531cb95165484017e356eb64b1846b3e2d4c7527d7c3290f90d50
Deleted: sha256:eb0cf91285f36c5036c44b8718a48cfe4ce1df1e0ce255afddca91d5de870d3f
Deleted: sha256:0c03333f85171ad015f6a61e42afb3640f636d93230e478d5aaf8c00b127132f
Deleted: sha256:aa3ad9d419d04f76e5e2b5c1aef31ddcfefe98a618821acb5fc303d6129b46d7
Deleted: sha256:a8200d1035fe769d9e72a6e82202428f28ad6fa114add05d09ac4b348297baa8
Deleted: sha256:944552b06886ab1272b49a7d8d85a902b9fda197c09b0b5dfb5183fa9dff89b3
Deleted: sha256:d4fb5654efd0d1eed5df3d117aad46b34e39742c2ef39a4fe2adf75a036352d4
Deleted: sha256:48103ae2dad538901243194a7391c6077036f7d06af47d69b901f5e4f0477bca
Deleted: sha256:36f82fbb4fe8bc838c960695011c51fc59cf1e53576b25193fd8396ef5476f25
Deleted: sha256:da4d65da834fe6b0715de7b186563ca5599042ec7ce96dc00ad3a13cdf0cb64c
Deleted: sha256:457c783df37685fae1e69aa3ee9307778c597b86e32f8165967c74bdc973112e
Deleted: sha256:85b420efc58448f3799c61873f3ab8f3746070beda86f7be8c2dd305de86d132
Deleted: sha256:82b279b4e759d1edcf9ee267d7a11749f1f35a8923a5e6b8abeabb5c23e7b8fb
Deleted: sha256:78545ee548ee5e8b8b1dd859f9e08bbcbd768998eac69ebdc48d10df16ce203b
Deleted: sha256:5ecc4be6ed9fe8e89c0c7facf5867ec2518f58eba85398a0d1a77f078fd9b710
Deleted: sha256:5d546a8805caf33cdaec21b1d59ec8fb8c161afb80a22fde57625cfba7835622
Deleted: sha256:d28ebaadf253f9c98586d904ab7ba4f5ed0a3efb9f302cb634cfee505fd024eb
Deleted: sha256:2b462408c8b99a1b997be57967410b2fae8e760d1496e169d16498b14e14d2c1
Deleted: sha256:264d982114935e37c0ce467ddb9ac79007b9ba137eccad3a57bd97a4d2aa1280
Deleted: sha256:431d5859e17b3dd8f25e63b1e20aa3055a7fc01699746170422ac72de8f793cd
Deleted: sha256:0a4ca592d7347850eaf48dedd8a2dab71a642302fede27b61fec0daad6837f4c
Deleted: sha256:586b8a8db57087455949773d93cda284cff4cb53b941eb0321c9c2ab8295040d
Deleted: sha256:4f34385c2e164af07af8c40f6d9ec047c0317b98dc2e01c239be434dbd492dc2
Deleted: sha256:af53335d7e0a9006b09d7dd4f614073d77aedb21d94a8a58bfe2586eeff8cd61
Deleted: sha256:2d29e06a7ea502ebd09b90657a0cefdbaad2bc9d1d6cb05df2463423e540c77f
Deleted: sha256:429d9f148dc6e2d2bd3241ccb0b21bbc74965519d68d948b60dad88fb638d0ef
Deleted: sha256:c3f226ec28aba2e42e0649bdeff71f89101fd1deae53738a598cfa0b1483df62
Deleted: sha256:495f8b348081c2d207c15f7868022b2d016d90739173e5b1d0ef9b1347911cf1
Deleted: sha256:4233e205914451d874e1135c46363821648eda9a34403a03892ef5e616b055f2
Deleted: sha256:7307a75b9a79ccae4e380262c0cff648d5f0f6c5774dc93d01e6e6a0d6c7a164
Deleted: sha256:fdc4000e9e4fd84551da9217eed4c45287f56eda02388c3af154b5084e8cfe6b
Deleted: sha256:bb944b5dfad405e6897cf92bf4a1ca4d0da2ec46b4938b8a639d6cd649feaac9
Deleted: sha256:91b9ec5c0adf41085ed9dfb7b2b05f3bceb716fc90ca0071db12f1db874a2790
Deleted: sha256:3cc46fbb62e40ef2f62e5ff27dae4e8c089a516ebda0c95603f2d3005bba7464
Deleted: sha256:8ceb062565089f330562908a251877a7e100294b9c6bb2420a68c088bef5141c
Deleted: sha256:4189b4a3e09f537739da68ff5cb3f95a543c9e7b103118a29fe1962c21ef67d9
Deleted: sha256:36bfba69bdf347d20ac46a605f14d2f37e135365ffc492e9c81860724893e29d
Deleted: sha256:15af6db94f5e3ba715459d9fe8371fa141ddfb3db665168e9fbea314b3a02d4e
Deleted: sha256:cb9dfb896aa0f61eb2d06004d3be56f912b521c4b434c75c3a61dc2d9f09e689
Deleted: sha256:e327bb1ee245cdbb14738d0bd5e0b405f619a1b5df96dca31f25cb935eaa324e
Deleted: sha256:e3583072983e41061fd93b73d32c93336398a26ddb5cda66bf7b5dccde69793c
Deleted: sha256:0315e78b3a89a219fd56925f9e0817ec1386e4b72b84b4c7adfb2f7c3fafc162
Deleted: sha256:145af2aec984064c5a33b945fcb87a8f03428a8d4bd8ee7eefb261f6e3f5ac84
Deleted: sha256:2d8e0c9962ae3743d5ec5d16b80f4e9292bb23afec070d5cd1b7fad90021780a
Deleted: sha256:b53768ddafc7d6c9c5536efef796316334dbba7bdcb4c22d63b05c03e139cb17
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:0840de1543ecd50b5e5a241b773c3697fa3fbea661d062ec59308946f2b754e2
Associated tags:
- 20210912-121903222604151
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-121903222604151
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-121903222604151].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:0840de1543ecd50b5e5a241b773c3697fa3fbea661d062ec59308946f2b754e2].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:2ec2e41063f5fc332cd992f10dd2d82e49a1931e030a7ea8b4c6cceb6b883fa7
Associated tags:
- 86b3af13-f7da-44f3-8dd6-5f22e6549ca0
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:86b3af13-f7da-44f3-8dd6-5f22e6549ca0].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:2ec2e41063f5fc332cd992f10dd2d82e49a1931e030a7ea8b4c6cceb6b883fa7].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:52df570b4d69b63037326aec2f4f8a99691b7e8cac05db9fcd34acd807c5a565
Associated tags:
- 399f07a2-de14-4645-a3f1-00561fdec01a
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:399f07a2-de14-4645-a3f1-00561fdec01a].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:52df570b4d69b63037326aec2f4f8a99691b7e8cac05db9fcd34acd807c5a565].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:d1436803432624d7a7de723df8e17a6db5d2aa2bb1bc2435243b5f87bd744c26
Associated tags:
- 20210912-121859154246788
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210912-121859154246788
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210912-121859154246788].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:d1436803432624d7a7de723df8e17a6db5d2aa2bb1bc2435243b5f87bd744c26].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:ae095052ffd52908f5a4e216addadc81d98721d0e6ecadeb0e6b358ff62acd5e
Associated tags:
- b0a41404-597c-4b8e-a1fb-92dc8a4278dd
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:b0a41404-597c-4b8e-a1fb-92dc8a4278dd].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:ae095052ffd52908f5a4e216addadc81d98721d0e6ecadeb0e6b358ff62acd5e].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:1fad6bf4c4906a586d16a78e623b9abf9b4549e5357374434df4295264c8ac59
Associated tags:
- 212b6adb-7991-4ee7-aa70-e8fd779b1228
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:212b6adb-7991-4ee7-aa70-e8fd779b1228].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:1fad6bf4c4906a586d16a78e623b9abf9b4549e5357374434df4295264c8ac59].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 51m 8s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nafbkd7dd6dca
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8237
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8237/display/redirect>
Changes:
------------------------------------------
[...truncated 1.20 MB...]
apache_beam/examples/wordcount_it_test.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:108: in _run_wordcount_it
run_wordcount(
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f04167270a0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f0416721700>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f0416721400>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
result = self.run_pipeline(experiment='beam_fn_api')
errors = metric_result_matchers.verify_all(
result.metrics().all_metrics(),
dataflow_exercise_metrics_pipeline.metric_matchers())
> self.assertFalse(errors, str(errors))
E AssertionError: "Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>\nActual MetricResults:\n" is not false : Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>
E Actual MetricResults:
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:60: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
==================== 4 failed, 3 warnings in 517.46 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-061445081831904
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:500dea318b70ac6c55ee57fa52d4731d9cc1e8b05d223956d4ef8b3717fdb7a2
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:a35797ba-6c76-4679-a7ea-89a1353a75ea
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:acbcab9a2b7e8573a1379fcf24e5e7cb7497383aadd4d6eabcb56b59e05e85c9
Deleted: sha256:960063bc9fc965dcbecf7d13526182aeddabb877b2bedd926caab9116c6c8917
Deleted: sha256:fe46e4f4d92cd4373736650e381a7d8ef6ea3da98d30def738667d55c30d67a6
Deleted: sha256:5380dd32c6b14566446f9c3c196615e008ea50504f013b106c044282f328e8d4
Deleted: sha256:be42ea2d6497799a5a16ce446adac8ba02e60eeac24c070a0bf45209fd906fc7
Deleted: sha256:4b931cf93ab24b6b592e9ba1ab3b828527b3c0869d9d246556014826e1d8b3da
Deleted: sha256:0da7bc4822e552b591601acdb32c5782d792d8e25bb90460c5a64980625e00c8
Deleted: sha256:0372dcd482c3941121cb9b10bd1f708821e5dd597e409e378e16ba15327f702f
Deleted: sha256:78f72aa15d8fa704f46f3faa645dd73ed366961ae5e625f1758c4f439fe9b9a1
Deleted: sha256:2597e0e0301ab2ee0ef31a64262d89d0495d51e22bbc2a77f5bcdf0f778bf1c8
Deleted: sha256:d95971db009f3c3e80c53c26c274b35b8e54837407ebdb4296d11041c4b02c88
Deleted: sha256:6a15c6e977243bef674e61fcd4cb3ceb9961d8f364470d9c76148e2f3d9ed7d1
Deleted: sha256:ba14313f4e6156a4f8af1833892736ff63b30b1b373d5ce3353281c44d3f2d23
Deleted: sha256:e90839037f9ce06b1c8d55f439bee0082d5c77a89cbf1d522a4033e212e787da
Deleted: sha256:5c1af3ac3bf1bfdbf737d604dc7d722888ee177af6f886f6a89ab061f45a5975
Deleted: sha256:99c336985cf3343fa257a9c932a03a52d58493d868d4674c353091ba28a02be9
Deleted: sha256:6abd7d82dfac044570aeb4c0555dddedbeb2b778ce6a11b272e1209584a4e0fc
Deleted: sha256:856a6a894ce867107c50df7ee4b46536fade004aeb67abd5dc37b8e82cebf850
Deleted: sha256:0707a0b88a77ac9c7661278cb4473ec629198d135879a4a8c882a3fee72c16fd
Deleted: sha256:76ec80c0a0a045689e68ad294b678016a0cf8e22017058bbcd1134e62e98f42b
Deleted: sha256:8617784b7944f51edf9eaf89112635b69d2e41f4443a058715a6a21f27f07bb6
Deleted: sha256:ed4f9a55a6c8f4dd5039eff15da4ebe9e1c5113ef079525eb0d2022a95966885
Deleted: sha256:218bb4697df7831f889126438095de17f668e3f1fd38cbbac0e07ad625563472
Deleted: sha256:1bcef6333c37a07b26349a24c1c49e05ed4a91f6faa50eb7a2202aec1fe0c13c
Deleted: sha256:b2a6cceaaaf7937f209b724dfac00598005bb67b543046f14b50f62120ec828c
Deleted: sha256:df09ce5175c4a5fb63b692366a077511de20ea7e5919ba1439b733842308bc91
Deleted: sha256:e2c75423070a92d0de87b6ab88e6b662f28af9fa730dbdb88ddcd448901c3a41
Deleted: sha256:9b38d2da127e3b1329ef614975a730b6857ecbc032e5b11e27dc0613e00109bc
Deleted: sha256:68aa61e87bd7a4ec5c7e0cafaf994edab91e150a2308594a3b51063a313f404c
Deleted: sha256:b93fb8d350f9923f7caa99d6384e64e7b283bcdcdff2212c75d48e6a81b5d8cd
Deleted: sha256:5d8a8fb49cb6052c502e7dd01f03bc9e1a3ced2b33059242fb98a92ad7750a51
Deleted: sha256:216c778e6899a8cde612cf7748b832949d1e15f9fd35051670929c4ebd80976f
Deleted: sha256:035b387241bc835446ef231884fa4285e9433ad486ad0e080a4de3d108c28b6a
Deleted: sha256:376d2796b99bac51d077de615e57483f2e7c5a5e77a8201598e4ff0be39c674d
Deleted: sha256:c1160bebbf7c11d351ac3c4f612720fd86ca569e1412d9785e5adec6b7b378a8
Deleted: sha256:e0398e536a3af9126ca8ba3fa92c75807fa19532a288afbd52ed79d0a09a4753
Deleted: sha256:dd7da90dac1161c469c3aff22e27b8b2950396854585558a2806f033837ac471
Deleted: sha256:fb42712407b9f226053227a3022b940269dde690d95b3efd92155beed5c3d3e5
Deleted: sha256:6cc3426f8c5c1e2da08f40c65abe933031aaf79cfbdc23d757b3eab25a487932
Deleted: sha256:171c4db3bf61f3ee2602fd9c80dff577fa318d20ddd33b851901632f08598059
Deleted: sha256:d5d1968348195c9ba9e7d4aa45325488f235b2dcd17af68eb078a7e00616ce61
Deleted: sha256:4961afd02a052ee5c42c1eb7e1d730c94fade529bbbc15fe60211e6e19fb04ce
Deleted: sha256:4c6547a54e9559c9b613c433c638ba44d59c3c7d4f19a3f2390624da8da4f027
Deleted: sha256:1a6ae0b734295b97f35322860da6799f5df6ea7e2a7ae48b77acf1fd3e84b4f4
Deleted: sha256:5f20a9dd87e4abbfc71118b4692b628bb50040aa46a510f367c9fc314812a229
Deleted: sha256:be305753fbc443e2c8c2a74d9b2206ba022068e509cf7d86cc0fb4a3642e2971
Deleted: sha256:6b7b8a4f859c1d2752b3d86075f2d15a3d5d4a117ba947638032838c5af216a1
Deleted: sha256:ff78e48b127fdf1d7581ec965c0ba3058a827575eedf96afe483e6858e6eef2b
Deleted: sha256:d49367cc471b052edf47b8e7e09b3b833dfe1473a36c711328645faae2501851
Deleted: sha256:5325ad5039a8c23ea0f337092ebbf9c39fa781b7a13eed56ba62f207473c84ef
Deleted: sha256:b505d4d0df6a36c9bf7576ab5d6c02188dbf90c5363c5e95514f0aafcf7a8b8b
Deleted: sha256:b333cc6a7c81e51bfe1668d6eb86c0febd3c8be2dba44802726d07cf60da6375
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:500dea318b70ac6c55ee57fa52d4731d9cc1e8b05d223956d4ef8b3717fdb7a2
Associated tags:
- 20210912-061445081831904
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-061445081831904
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-061445081831904].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:500dea318b70ac6c55ee57fa52d4731d9cc1e8b05d223956d4ef8b3717fdb7a2].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:acbcab9a2b7e8573a1379fcf24e5e7cb7497383aadd4d6eabcb56b59e05e85c9
Associated tags:
- a35797ba-6c76-4679-a7ea-89a1353a75ea
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:a35797ba-6c76-4679-a7ea-89a1353a75ea].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:acbcab9a2b7e8573a1379fcf24e5e7cb7497383aadd4d6eabcb56b59e05e85c9].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9c624017e1a1e52d01f5b41b4747c3dd564f765284038614738fe26ea606a0a3
Associated tags:
- 96b361b5-b84e-45ac-9c3c-6ebb8942618a
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:96b361b5-b84e-45ac-9c3c-6ebb8942618a].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9c624017e1a1e52d01f5b41b4747c3dd564f765284038614738fe26ea606a0a3].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:83b4d74d8919a86c66dd783b216ff2018f4dd96c559d39572c6a11f97c13f67e
Associated tags:
- 20210912-061433748967836
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210912-061433748967836
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210912-061433748967836].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:83b4d74d8919a86c66dd783b216ff2018f4dd96c559d39572c6a11f97c13f67e].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a024e6a8de53120df291045e63ee8672d3d3c93e33df51e59a5fbd89c66048c0
Associated tags:
- c90abe90-2a68-4c7a-8810-f2be64ced36e
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:c90abe90-2a68-4c7a-8810-f2be64ced36e].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a024e6a8de53120df291045e63ee8672d3d3c93e33df51e59a5fbd89c66048c0].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:9a663c2497e9ddfe143d9ae4aea89b28834308d85fc02619307733aa4970c091
Associated tags:
- e98a8512-f37e-4f66-85ca-71874d6c894a
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:e98a8512-f37e-4f66-85ca-71874d6c894a].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:9a663c2497e9ddfe143d9ae4aea89b28834308d85fc02619307733aa4970c091].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 6s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wenj24pxgtk74
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8236
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8236/display/redirect>
Changes:
------------------------------------------
[...truncated 124.08 KB...]
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
> result = self.run_pipeline(experiment='beam_fn_api')
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:43: in run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py:176: in apply_and_run
result = pipeline.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210912002232041030-5768'
createTime: '2021-09-12T00:22:33.348250Z'
...021-09-12T00:22:33.348250Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7fd27ce07400>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
> raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1632: DataflowRuntimeException
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
==================== 4 failed, 3 warnings in 482.84 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-001421369868966
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:ef0e09b0f634a355c243e6c698b0225cc14df729faa86f1d262e1e25a5a4178c
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:247fd4ad-d696-4550-a2d8-1e7a469fcc74
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:a3b2c61828581b6fe2759ee28a719cbb0acd34ab35930b27bb37f7f09dabea49
Deleted: sha256:c83cdae7a1313b816f2431ccc23b9b2df8b76558f00726b26b51fa7391d12b58
Deleted: sha256:2ef176ae5d77506c363c6f8e495b34598402d988c99c179883b40ee9b6186ed9
Deleted: sha256:b637ffa693f2369372d3f54189340a9bd92d5d37748c8e64ff1fbc5e57cc2ca0
Deleted: sha256:f4caffa1a451c27cdd986796ac823f82cc6464f8477b4c7551408789dc18675e
Deleted: sha256:2b97d51cc523552faf35ac704dc36ae03afb238095a219bd8c92d16cd689cba2
Deleted: sha256:aa6cf5fc68671a1cadd6355318f77776744654edba1dda13bf0aaee8656e957f
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:ef0e09b0f634a355c243e6c698b0225cc14df729faa86f1d262e1e25a5a4178c
Associated tags:
- 20210912-001421369868966
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-001421369868966
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210912-001421369868966].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:ef0e09b0f634a355c243e6c698b0225cc14df729faa86f1d262e1e25a5a4178c].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:a3b2c61828581b6fe2759ee28a719cbb0acd34ab35930b27bb37f7f09dabea49
Associated tags:
- 247fd4ad-d696-4550-a2d8-1e7a469fcc74
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:247fd4ad-d696-4550-a2d8-1e7a469fcc74].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:a3b2c61828581b6fe2759ee28a719cbb0acd34ab35930b27bb37f7f09dabea49].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:51e3f23910adf6a08ee3ce194ebe065971437ab4eef595e67318a25de95c0905
Associated tags:
- 9055bce4-08d2-4518-b2a5-fecbba058051
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:9055bce4-08d2-4518-b2a5-fecbba058051].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:51e3f23910adf6a08ee3ce194ebe065971437ab4eef595e67318a25de95c0905].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:176092d6b02ef70bb7e0e0fa882a7940d427db72466cd7135382f5c9ff6e44f5
Associated tags:
- 20210912-001421498675863
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210912-001421498675863
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210912-001421498675863].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:176092d6b02ef70bb7e0e0fa882a7940d427db72466cd7135382f5c9ff6e44f5].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:590aef15ea6a1c3389ac92dc14844f4fd2046288f904b03879ed016c20f18049
Associated tags:
- c48a91ae-e955-4bb6-93a6-472e12359e37
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:c48a91ae-e955-4bb6-93a6-472e12359e37].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:590aef15ea6a1c3389ac92dc14844f4fd2046288f904b03879ed016c20f18049].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:bcf856e26453069a61ea96fbe09fae6dc133b6f0f369ef4f99586a5122d7574c
Associated tags:
- 288f8aa8-aec5-4cb2-84e2-0f2c3870ab80
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:288f8aa8-aec5-4cb2-84e2-0f2c3870ab80].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:bcf856e26453069a61ea96fbe09fae6dc133b6f0f369ef4f99586a5122d7574c].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
[gw0] [ 50%] PASSED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
[gw0] [ 75%] PASSED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
[gw0] [100%] PASSED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml> -
=================== 4 passed, 2 warnings in 2020.04 seconds ====================
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210912-001421469578424
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:9fa5e941834cbdb041d19125609b2964f18e1e3851a9a05d8ce5a31016792a6f
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:afb58967-2118-4b7d-b91d-0cf3c7da113b
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:6b4186250067ae2e466f57376d8dccd76d3b020a6af5a086c21e20b9ba40f8ae
Deleted: sha256:c68be6663be28de0cfb68466e2cfcf498da8e03af1e0ccf08f20f9d57616b163
Deleted: sha256:81e2c59bac27e641db3d4488994480fea8fb6afa68e2af70fe1b6089cf77f40a
Deleted: sha256:279634064d178ad954410ad1f509fbaff035d931cad4b60fbbadb441321dda86
Deleted: sha256:adf447a0241b9e08449960a2707e00bfbb78510f4de12304b4b2ebe38dd09d2d
Deleted: sha256:9899f8e8a33f7d92f3d29506f6d0e751b731b82af072a7060ff6be1c74b71c1e
Deleted: sha256:8f432af6817fa8d876b0019da15f319bc0fe2cfa3125dfcfbbd1829af3930a90
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:9fa5e941834cbdb041d19125609b2964f18e1e3851a9a05d8ce5a31016792a6f
Associated tags:
- 20210912-001421469578424
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210912-001421469578424
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210912-001421469578424].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:9fa5e941834cbdb041d19125609b2964f18e1e3851a9a05d8ce5a31016792a6f].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:6b4186250067ae2e466f57376d8dccd76d3b020a6af5a086c21e20b9ba40f8ae
Associated tags:
- afb58967-2118-4b7d-b91d-0cf3c7da113b
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:afb58967-2118-4b7d-b91d-0cf3c7da113b].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:6b4186250067ae2e466f57376d8dccd76d3b020a6af5a086c21e20b9ba40f8ae].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:bbff8b932523f9ff054c80fab330e4be1dce32dc761a9fefaec2673692f3fb87
Associated tags:
- 5d54961c-d9cc-440c-abab-87f2b55523cd
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:5d54961c-d9cc-440c-abab-87f2b55523cd].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:bbff8b932523f9ff054c80fab330e4be1dce32dc761a9fefaec2673692f3fb87].
Removed the container
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 49m 39s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/roppwbqingc3k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8235
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8235/display/redirect>
Changes:
------------------------------------------
[...truncated 142.37 KB...]
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:f1904e3ec9d1b9f4d38bc5d7d018ce6baed3b1f501dedd3c97f88ef933942c29].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
[gw0] [ 75%] PASSED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
[gw0] [100%] FAILED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=================================== FAILURES ===================================
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_cloud_build _____
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_cloud_build>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self):
> self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:64:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:108: in _run_wordcount_it
run_wordcount(
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f277a93efa0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f2775efdd30>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f277ae39d90>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
> result = self.run_pipeline(experiment='beam_fn_api')
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:43: in run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py:176: in apply_and_run
result = pipeline.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210911183806875583-3474'
createTime: '2021-09-11T18:38:08.293216Z'
...021-09-11T18:38:08.293216Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f277a92f520>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
> raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1632: DataflowRuntimeException
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
============== 2 failed, 2 passed, 3 warnings in 1356.06 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-181517537871768
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:366b9fffd1e0eec652d4c515e26a2217617a69dd0295e84a57303dc0c863b048
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:0b4b3ccb-38c1-40ac-b19b-2b89ffd42d59
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:128aac4280d929438eaf8c123c68939c32f2f0d59949e46af2c52012ef2f54c6
Deleted: sha256:fec264ee7cbfd3918e275fd33e56063d15234e733421a0b37c989fa11a908682
Deleted: sha256:e080a4f009954c95b3658469b71a7939003e65270b22f0eca045f484acad120b
Deleted: sha256:a2aef753aa1faea627a901977d539e8a85cb00ecea0ee6ba306fbbc5ec5fa593
Deleted: sha256:edfcfc191dd33906daeb22f34a8b1ed48fe5b44d9eed2d3987e08eea1aca987d
Deleted: sha256:8a65aed4a3fe91fe603a9fd166b9b17898d8e3d0d9311f5dd96f8dc7ed973e9f
Deleted: sha256:a1c4bcc7b99936925c3d4c78ae3aaab5ca58fb45557a4c7c1e822f0be3c74b7c
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:366b9fffd1e0eec652d4c515e26a2217617a69dd0295e84a57303dc0c863b048
Associated tags:
- 20210911-181517537871768
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-181517537871768
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-181517537871768].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:366b9fffd1e0eec652d4c515e26a2217617a69dd0295e84a57303dc0c863b048].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:128aac4280d929438eaf8c123c68939c32f2f0d59949e46af2c52012ef2f54c6
Associated tags:
- 0b4b3ccb-38c1-40ac-b19b-2b89ffd42d59
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:0b4b3ccb-38c1-40ac-b19b-2b89ffd42d59].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:128aac4280d929438eaf8c123c68939c32f2f0d59949e46af2c52012ef2f54c6].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:1bbbc2813c979c0b87c00a69fb3325949a4dd09c1c92ae5a11dbc8180b0dd14b
Associated tags:
- 25a40960-88db-41d6-b71c-ca6720d53f80
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:25a40960-88db-41d6-b71c-ca6720d53f80].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:1bbbc2813c979c0b87c00a69fb3325949a4dd09c1c92ae5a11dbc8180b0dd14b].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 39m 25s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/m4f53ilplx7qw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8234
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8234/display/redirect>
Changes:
------------------------------------------
[...truncated 118.99 KB...]
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:542: in run
self._options).run(False)
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f1daf50be80>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f1dae7a7828>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f1dae6f62e8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
result = self.run_pipeline(experiment='beam_fn_api')
errors = metric_result_matchers.verify_all(
result.metrics().all_metrics(),
dataflow_exercise_metrics_pipeline.metric_matchers())
> self.assertFalse(errors, str(errors))
E AssertionError: "Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>\nActual MetricResults:\n" is not false : Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'total_values' step: 'metrics' attempted: <100> committed: <100>Unable to match metrics for matcher name: 'ExecutionTime_StartBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_ProcessElement' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ExecutionTime_FinishBundle' step: 'metrics' attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher namespace: 'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn' name: 'distribution_values' step: 'metrics' attempted: sum_value: <100> count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100> count_value: <4> min_value: <0> max_value: <100>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'metrics-out0'). (label_key: 'original_name' label_value: 'metrics-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'map_to_common_key-out0'). (label_key: 'original_name' label_value: 'map_to_common_key-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/GroupByWindow-out0'). (label_key: 'original_name' label_value: 'GroupByKey/GroupByWindow-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Read-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Read-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'ElementCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-ElementCount'). attempted: a value greater than <0> committed: a value greater than <0>Unable to match metrics for matcher name: 'MeanByteCount' (label_key: 'output_user_name' label_value: 'GroupByKey/Reify-out0'). (label_key: 'original_name' label_value: 'GroupByKey/Reify-out0-MeanByteCount'). attempted: a value greater than <0> committed: a value greater than <0>
E Actual MetricResults:
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:60: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml> -
==================== 4 failed, 2 warnings in 459.20 seconds ====================
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210911-121231205434773
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:edefc00cf9e1c2caf2a9c81e99329ade7f98a45a24795f6a01578fbaa191ab0a
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:aa0eff93-970e-45de-bb1a-6ffdaa0e14a5
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:e832e5aaecdd265902b9c8754b736119c7f47317c9de931efa28cf9c4a99f8c1
Deleted: sha256:ae82cf368c1d653c1b698fe0d5c96c3d7d56310317ce708df350d49e0cde3f01
Deleted: sha256:5a2ada9d79812a5eb98ec85827cba8e47b1a2752cc93f7e7934005f93587bbf6
Deleted: sha256:2bfc524169fb47c7c2c0abda08f31c5bec0517b4b1af15e527732e9c7efefb27
Deleted: sha256:0906467901e10fd8e6c85f2ca081c73db2d3630eb0f0052817c6e2d88f5fea08
Deleted: sha256:f334acfd6c2c0987698cb2e0b0e363553e59cb471520d1e32385349ea34bb6f9
Deleted: sha256:5af836e8ab2f7b101c60a20232172ff7a41ec995c0b64b86eace5156f10e2d78
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
[gw0] [ 75%] FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:edefc00cf9e1c2caf2a9c81e99329ade7f98a45a24795f6a01578fbaa191ab0a
Associated tags:
- 20210911-121231205434773
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210911-121231205434773
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210911-121231205434773].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:edefc00cf9e1c2caf2a9c81e99329ade7f98a45a24795f6a01578fbaa191ab0a].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:e832e5aaecdd265902b9c8754b736119c7f47317c9de931efa28cf9c4a99f8c1
Associated tags:
- aa0eff93-970e-45de-bb1a-6ffdaa0e14a5
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:aa0eff93-970e-45de-bb1a-6ffdaa0e14a5].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:e832e5aaecdd265902b9c8754b736119c7f47317c9de931efa28cf9c4a99f8c1].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:f2d4ba2919b312c363c1de82279f78294248e88628563c6000d396e2e4654e77
Associated tags:
- 682b9cb0-2f88-40b4-a09b-88265d24d27a
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:682b9cb0-2f88-40b4-a09b-88265d24d27a].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:f2d4ba2919b312c363c1de82279f78294248e88628563c6000d396e2e4654e77].
Removed the container
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:22b1cf1ec7d0383464127bd70636d79a8493a600c25e0b3b14af1aa3225ca8fb
Associated tags:
- 20210911-121230864096745
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210911-121230864096745
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210911-121230864096745].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:22b1cf1ec7d0383464127bd70636d79a8493a600c25e0b3b14af1aa3225ca8fb].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:989eec10ce25b441f696bf7ecb38fe906fbdf5261126ad1bf6f32958081ffdfb
Associated tags:
- aa5fbd5c-540c-44d6-9419-63ebe5030d5d
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:aa5fbd5c-540c-44d6-9419-63ebe5030d5d].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:989eec10ce25b441f696bf7ecb38fe906fbdf5261126ad1bf6f32958081ffdfb].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:59274894030679a2e626ab2340bc0765eebff6073a971225a8766375a04b711c
Associated tags:
- ca524d24-d037-467c-89a3-86ff528f3554
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:ca524d24-d037-467c-89a3-86ff528f3554].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:59274894030679a2e626ab2340bc0765eebff6073a971225a8766375a04b711c].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:4c3e050ad1038855a722326083e4be4de10669b961cc432123500e459cce8ef4
Associated tags:
- 20210911-121240163256728
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-121240163256728
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-121240163256728].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:4c3e050ad1038855a722326083e4be4de10669b961cc432123500e459cce8ef4].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:ef4b33a3baa368964b2553820b0779fb6a56e321794609c7a0d95c68f0f39d24
Associated tags:
- 371f6431-24b8-40c1-8a29-ea970f275f71
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:371f6431-24b8-40c1-8a29-ea970f275f71].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:ef4b33a3baa368964b2553820b0779fb6a56e321794609c7a0d95c68f0f39d24].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:489f8d4970052cf9f72b585ff30284cad8857bab400b64bfce4a1b88980d7f85
Associated tags:
- 40c7db2c-9eca-4cca-89a5-7084ffad68ea
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:40c7db2c-9eca-4cca-89a5-7084ffad68ea].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:489f8d4970052cf9f72b585ff30284cad8857bab400b64bfce4a1b88980d7f85].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 28m 28s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4tl54xqt4u6nw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8233
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8233/display/redirect>
Changes:
------------------------------------------
[...truncated 107.73 KB...]
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:e9d6262ac2a2b75ba435bc8d28104b4a1bbc2ef97da790cb753336b3d6759b69].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:ea8d1123c2ab66bc9b1581a5a483a51187723b4f654df7c9c89a58ec60d93590
Associated tags:
- 7b911cf4-d22f-47e0-b993-695b1e8ecc72
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:7b911cf4-d22f-47e0-b993-695b1e8ecc72].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:ea8d1123c2ab66bc9b1581a5a483a51187723b4f654df7c9c89a58ec60d93590].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
[gw0] [100%] PASSED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=================================== FAILURES ===================================
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_cloud_build _____
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_cloud_build>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self):
> self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:64:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:108: in _run_wordcount_it
run_wordcount(
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f33a0ce7b80>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f339c78abb0>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f33a0ce5c70>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self):
> self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:108: in _run_wordcount_it
run_wordcount(
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:587: in __exit__
self.result = self.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7f339c715bb0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7f3399e71310>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7f3399ce2d90>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
============== 2 failed, 2 passed, 3 warnings in 1216.35 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-061436898183517
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:c7827408e4a0ce966da946d5604c2ccb933648513a110cf42067b739fbaac4b7
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:827627d1-f349-4701-adaf-6c1d0b389a42
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:7c938fca97930be98bf8c590aa7329c97543c55b9bbb96e9ad9e89fd52287ffa
Deleted: sha256:d0796f4da40fef900c9955974b18f8c4d26bb8cd45f449a32d76f62129a27826
Deleted: sha256:87a54b915967b9a36ab7f78648dfac85fcb722796eb63a6b790e898fc98b2ea2
Deleted: sha256:1603b7d53e4c22b49c201935bb79c1d62b3f8161670a94520cd3e1fd8c06e83a
Deleted: sha256:df30e8ecab7526fa8ee84ecb5a43286ab073560489974e473d291f642d0d9c40
Deleted: sha256:aa3d1f64794d7b07b57ea4083c6d349a3fe37a941b71fa8dfe6b227d2fda7249
Deleted: sha256:af36883c11e11487eebd8985491e12837460a1f62fc0468e2d5667dfd441fa0d
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:c7827408e4a0ce966da946d5604c2ccb933648513a110cf42067b739fbaac4b7
Associated tags:
- 20210911-061436898183517
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-061436898183517
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-061436898183517].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:c7827408e4a0ce966da946d5604c2ccb933648513a110cf42067b739fbaac4b7].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:7c938fca97930be98bf8c590aa7329c97543c55b9bbb96e9ad9e89fd52287ffa
Associated tags:
- 827627d1-f349-4701-adaf-6c1d0b389a42
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:827627d1-f349-4701-adaf-6c1d0b389a42].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:7c938fca97930be98bf8c590aa7329c97543c55b9bbb96e9ad9e89fd52287ffa].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:1293e9b4f4484f1880e01538bd967dc2d462b411c8d866a7bc9d91e68faaef8c
Associated tags:
- 7c81542a-fcf1-4b33-a7f1-3544234062b6
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:7c81542a-fcf1-4b33-a7f1-3544234062b6].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:1293e9b4f4484f1880e01538bd967dc2d462b411c8d866a7bc9d91e68faaef8c].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 36m 19s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/o7xhdvbznzkcw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8232
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8232/display/redirect?page=changes>
Changes:
[noreply] Port changes from Pub/Sub Lite to beam (#15418)
[heejong] [BEAM-12805] Fix XLang CombinePerKey test by explicitly assigning the
[BenWhitehead] [BEAM-8376] Google Cloud Firestore Connector - Add handling for
[noreply] Decreasing peak memory usage for beam.TupleCombineFn (#15494)
[noreply] [BEAM-12802] Add support for prefetch through data layers down through
[noreply] [BEAM-11097] Add implementation of side input cache (#15483)
------------------------------------------
[...truncated 117.17 KB...]
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210911002907951086-7153'
createTime: '2021-09-11T00:29:09.493383Z'
...021-09-11T00:29:09.493383Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7fec863b42b0>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
> raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1632: DataflowRuntimeException
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
> result = self.run_pipeline(experiment='beam_fn_api')
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:43: in run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py:176: in apply_and_run
result = pipeline.run()
apache_beam/pipeline.py:539: in run
return Pipeline.from_runner_api(
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210911003050192824-1145'
createTime: '2021-09-11T00:30:51.423013Z'
...021-09-11T00:30:51.423013Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7fec8fa45e80>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
> raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1632: DataflowRuntimeException
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
============== 3 failed, 1 passed, 3 warnings in 1037.64 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-001238518901961
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:9b8b2415fea99c6c2522257a09dfdccc579407e300c324a99fe20a713b66cc23
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:d9c69f16-4445-4caf-8a8c-537f40c54f3a
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:4869bea8a39734edd1d301b1e67c4b9d78f109be3f892c8055985e07319f64e9
Deleted: sha256:cd4bc9ba385ad1db439c8f433510b138a39145525a681932719db73fcececbc2
Deleted: sha256:1ec5539dc175f15e59df2d0485b653e864ec08ba454bde4ee3fb241d83905843
Deleted: sha256:6c08810988e19af2d506d456c94d6d5495b5291c17b86eed56488468f30d5b44
Deleted: sha256:027b1440360b14bd7a9208d0a3e620591edf7ea56573abf8f1d0cf46a2322b71
Deleted: sha256:e8b90a221c5cedf21ad25bef7920d53449f2060dadd14d94b963152b25555093
Deleted: sha256:b08a1a1d8c9fea7b5fbd4d2a607e9c8ceedcbf1606330a8d5937aaea9dade966
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:9b8b2415fea99c6c2522257a09dfdccc579407e300c324a99fe20a713b66cc23
Associated tags:
- 20210911-001238518901961
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-001238518901961
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210911-001238518901961].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:9b8b2415fea99c6c2522257a09dfdccc579407e300c324a99fe20a713b66cc23].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:4869bea8a39734edd1d301b1e67c4b9d78f109be3f892c8055985e07319f64e9
Associated tags:
- d9c69f16-4445-4caf-8a8c-537f40c54f3a
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:d9c69f16-4445-4caf-8a8c-537f40c54f3a].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:4869bea8a39734edd1d301b1e67c4b9d78f109be3f892c8055985e07319f64e9].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:97a9cf9bef483a09678cc0a9c355e32b0c7337275485c0f20a718b7b8bcaaec3
Associated tags:
- 64122b35-9b27-4c0d-8213-8868570ef8a3
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:64122b35-9b27-4c0d-8213-8868570ef8a3].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:97a9cf9bef483a09678cc0a9c355e32b0c7337275485c0f20a718b7b8bcaaec3].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:3ee6ffb5e118ca444d6a83f85c726fcfef0036c1a5237f797482acfe9f935eec
Associated tags:
- 20210911-001229867151344
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210911-001229867151344
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210911-001229867151344].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:3ee6ffb5e118ca444d6a83f85c726fcfef0036c1a5237f797482acfe9f935eec].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:84ef1662660d37b7f1705b398654a3fa878fd878590a2e9b1c55102a380ea711
Associated tags:
- d372e381-13b6-43e2-990c-7d6caee0df51
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:d372e381-13b6-43e2-990c-7d6caee0df51].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:84ef1662660d37b7f1705b398654a3fa878fd878590a2e9b1c55102a380ea711].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1d2b0ee8c25227f7d39412f17b47f6c4871a4175e534bcf47b5c2b5c40f51681
Associated tags:
- efb71066-402f-4734-af28-5b2613d944ad
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:efb71066-402f-4734-af28-5b2613d944ad].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:1d2b0ee8c25227f7d39412f17b47f6c4871a4175e534bcf47b5c2b5c40f51681].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 41m 54s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/44eyrl5i3urf4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org