You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/11/08 17:39:25 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #9914
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9914/display/redirect>
Changes:
------------------------------------------
[...truncated 1.82 MB...]
E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
E at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
E at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
E at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
E at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
E ... 17 more
apache_beam/runners/dataflow/dataflow_runner.py:1592: DataflowRuntimeException
=============================== warnings summary ===============================
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/typehints/pandas_type_compatibility_test.py:66
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:66: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
}).set_index(pd.Int64Index(range(123, 223), name='an_index')),
apache_beam/typehints/pandas_type_compatibility_test.py:89
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:89: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(123, 223), name='an_index'),
apache_beam/typehints/pandas_type_compatibility_test.py:90
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(475, 575), name='another_index'),
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.9_sdk.xml> -
=========================== short test summary info ============================
FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
======= 2 failed, 2 passed, 5 skipped, 6 warnings in 2281.50s (0:38:01) ========
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221108-165619285052724
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:bfc17a4a1aeb0cd655cc8cc5d8a987f1c40caa034f39d658d6ab1e95f1b6baa3
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:071b9f93-6025-4cf4-b8c7-61bcf8a868b8
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:4415df405e9dd52ab9715ee91fa62e33b51b215cc8142ecb8f4ace4bd4fc9733
Deleted: sha256:065b5832bf22a7f4cc9412b9fb0d31006ab903d3fdf851c114525c6ec2101d8d
Deleted: sha256:f521e2c2ac1f55e0a2d1fca2e1f1da50dbc2798f54c774ecb6d054b13c670787
Deleted: sha256:e15216c6ffd9e17e122a0cc5249c9d7eff4f8318b29c1ce44ab8c4012d278d2d
Deleted: sha256:d9de2831db9a432303f1e9a2e6af3e479f401be5ef31a23056e8512ecdca5d31
Deleted: sha256:abbd30e05759800f1a31b293b49dfe6507f2b58c99a53d96affcb5a5ba76f081
Deleted: sha256:8e6d15b8099f734198cdce361c799888225338d3594039ffeb140b5806c3001a
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:bfc17a4a1aeb0cd655cc8cc5d8a987f1c40caa034f39d658d6ab1e95f1b6baa3
Associated tags:
- 20221108-165619285052724
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221108-165619285052724
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221108-165619285052724].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:bfc17a4a1aeb0cd655cc8cc5d8a987f1c40caa034f39d658d6ab1e95f1b6baa3].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:4415df405e9dd52ab9715ee91fa62e33b51b215cc8142ecb8f4ace4bd4fc9733
Associated tags:
- 071b9f93-6025-4cf4-b8c7-61bcf8a868b8
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:071b9f93-6025-4cf4-b8c7-61bcf8a868b8].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:4415df405e9dd52ab9715ee91fa62e33b51b215cc8142ecb8f4ace4bd4fc9733].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:5deb5c89e2d5baa912d76ae3104edf7656f76b371f35ca3a467d1b13091d2ec0
Associated tags:
- 035f8765-9fc6-4882-8a72-9cbba3ec5583
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:035f8765-9fc6-4882-8a72-9cbba3ec5583].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:5deb5c89e2d5baa912d76ae3104edf7656f76b371f35ca3a467d1b13091d2ec0].
Removed the container
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:ed2ff3d5460c5c6b4bfecb50f1356ee3d6c45ad68c7fa01d00d5e0cecae42564
Associated tags:
- 20221108-165619357819870
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221108-165619357819870
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221108-165619357819870].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:ed2ff3d5460c5c6b4bfecb50f1356ee3d6c45ad68c7fa01d00d5e0cecae42564].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:ac0f602d1ee2440e5393d571a33c0bf178f1a14f499179ffe57ec6caef908cce
Associated tags:
- 1a3e4cbf-7185-49b7-b598-ce445a7acd81
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:1a3e4cbf-7185-49b7-b598-ce445a7acd81].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:ac0f602d1ee2440e5393d571a33c0bf178f1a14f499179ffe57ec6caef908cce].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:26a1a4bf99503b6e15154b37e1b6b553b304a255478873aa3854e1cc3573bdc0
Associated tags:
- 20b4c4f5-2465-459c-8057-12ef69fdb179
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:20b4c4f5-2465-459c-8057-12ef69fdb179].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:26a1a4bf99503b6e15154b37e1b6b553b304a255478873aa3854e1cc3573bdc0].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py310:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 58m 45s
62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/habdm4tk2orf6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #9918
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9918/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #9917
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9917/display/redirect?page=changes>
Changes:
[noreply] [Go SDK] S3 implementation of the Beam filesystem (#23992)
------------------------------------------
[...truncated 1.97 MB...]
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/typehints/pandas_type_compatibility_test.py:66
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:66: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
}).set_index(pd.Int64Index(range(123, 223), name='an_index')),
apache_beam/typehints/pandas_type_compatibility_test.py:89
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:89: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(123, 223), name='an_index'),
apache_beam/typehints/pandas_type_compatibility_test.py:90
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(475, 575), name='another_index'),
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
=========================== short test summary info ============================
FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
======= 1 failed, 3 passed, 5 skipped, 6 warnings in 2221.82s (0:37:01) ========
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221109-105432540377646
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:b5d26b6ed9d074a2903ccecc0239f00668856501173f3fb6141ea3c76f318033
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:5fbc40e5-ae64-4cee-b9f3-7d47dea3095d
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:6f478d0b830b67996b8148c189a9fa7483f0aaeb552364d4b70d0a3ca5994bc9
Deleted: sha256:eb128ef280963285610101277217c460a505e2a13cac2cb4a738135b49cd8827
Deleted: sha256:67b4b5ebd4212fcab2859bf9cf78c71427e922c5b10c999d1b1bab1fd9eeddd4
Deleted: sha256:8d81892eb58b5c82229b0433362ab6e4579203391d921390b47531633ab5aa6b
Deleted: sha256:db6d4da85f0d0bb4fccb7fbc65e35d8f494d5fa8b8c205e72b80b9bc33daf233
Deleted: sha256:60351ebbf56f9e303bea4cf65a52252e32186fa5b817d69090f46ff651467665
Deleted: sha256:2afbacc86d2eb82a64487fb8d34228e40a28c4f4c8493cb3300c925b00279048
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:b5d26b6ed9d074a2903ccecc0239f00668856501173f3fb6141ea3c76f318033
Associated tags:
- 20221109-105432540377646
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221109-105432540377646
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221109-105432540377646].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:b5d26b6ed9d074a2903ccecc0239f00668856501173f3fb6141ea3c76f318033].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:6f478d0b830b67996b8148c189a9fa7483f0aaeb552364d4b70d0a3ca5994bc9
Associated tags:
- 5fbc40e5-ae64-4cee-b9f3-7d47dea3095d
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:5fbc40e5-ae64-4cee-b9f3-7d47dea3095d].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:6f478d0b830b67996b8148c189a9fa7483f0aaeb552364d4b70d0a3ca5994bc9].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:e5669228650b21fa9606e362b2fdb2996a00954eef374e6c8787c4917b25cecb
Associated tags:
- 524851d8-56db-4d5e-8c28-0f469b7a1306
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:524851d8-56db-4d5e-8c28-0f469b7a1306].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:e5669228650b21fa9606e362b2fdb2996a00954eef374e6c8787c4917b25cecb].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:412394c1502660597533ebfde3ca3ae136fed5e26e34233886de5cfdb17dc1ee
Associated tags:
- 20221109-105432446803329
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221109-105432446803329
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221109-105432446803329].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:412394c1502660597533ebfde3ca3ae136fed5e26e34233886de5cfdb17dc1ee].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:92f410555a9ac7a9d1227a48db74020ead52b468797de16fc93e0ad04b6cfd5c
Associated tags:
- 58f54b21-90fd-4a02-9277-904ff85f9340
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:58f54b21-90fd-4a02-9277-904ff85f9340].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:92f410555a9ac7a9d1227a48db74020ead52b468797de16fc93e0ad04b6cfd5c].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:da5f2a9a46851c368a9235a9ae8252123d8127e309e232f526b3b761d22b4e4a
Associated tags:
- 936a832c-06fe-4cd7-a2d6-4918c8de4a2b
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:936a832c-06fe-4cd7-a2d6-4918c8de4a2b].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:da5f2a9a46851c368a9235a9ae8252123d8127e309e232f526b3b761d22b4e4a].
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:ffb8e02137ab693510baa5a310e8f6f386defbdfa4493d2cfb3120d0daaf7aa4
Associated tags:
- 20221109-105432728813565
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221109-105432728813565
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221109-105432728813565].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:ffb8e02137ab693510baa5a310e8f6f386defbdfa4493d2cfb3120d0daaf7aa4].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:5ca72a19f2745885ebd09933604b0d7727067a5bab5435a741fb3b4df8dcbbca
Associated tags:
- fcda9270-1ffc-4a3c-92c4-acf2303a922e
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:fcda9270-1ffc-4a3c-92c4-acf2303a922e].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:5ca72a19f2745885ebd09933604b0d7727067a5bab5435a741fb3b4df8dcbbca].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:7eea0e409c897b16783d67c5aff894c74b67b0a5d228d1fb03e434abfca4c019
Associated tags:
- b4786e65-63f4-4d31-a7dd-3ed3eb114d35
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:b4786e65-63f4-4d31-a7dd-3ed3eb114d35].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:7eea0e409c897b16783d67c5aff894c74b67b0a5d228d1fb03e434abfca4c019].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:1d552e1b5dcc7e3c62b7e6c1f3697a21641390f488387296a30bebdabdf2d947
Associated tags:
- 20221109-105432785116958
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221109-105432785116958
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221109-105432785116958].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:1d552e1b5dcc7e3c62b7e6c1f3697a21641390f488387296a30bebdabdf2d947].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:a70d108a7da7a095da54dd09426adbf887b3c25d477e1d47fa40321c540419a7
Associated tags:
- 174206c6-d6ab-4cf4-a44b-4e02187a1d33
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:174206c6-d6ab-4cf4-a44b-4e02187a1d33].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:a70d108a7da7a095da54dd09426adbf887b3c25d477e1d47fa40321c540419a7].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:39d9bab64c00d87e0e139a86be037f0a9d6f499a628d4b3fd4062d65eaabe0de
Associated tags:
- 4cb4dc85-c587-4a4e-9276-07ccf2dbebb5
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:4cb4dc85-c587-4a4e-9276-07ccf2dbebb5].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:39d9bab64c00d87e0e139a86be037f0a9d6f499a628d4b3fd4062d65eaabe0de].
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py310:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 56m 26s
62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/4fxuifgivypfm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #9916
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9916/display/redirect?page=changes>
Changes:
[noreply] [Go] Add pipeline resource hints to CHANGES.md (#24036)
[noreply] Handle Avro schema generation for logical data types in
------------------------------------------
[...truncated 1.87 MB...]
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py:15
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/typehints/pandas_type_compatibility_test.py:66
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:66: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
}).set_index(pd.Int64Index(range(123, 223), name='an_index')),
apache_beam/typehints/pandas_type_compatibility_test.py:89
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:89: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(123, 223), name='an_index'),
apache_beam/typehints/pandas_type_compatibility_test.py:90
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(475, 575), name='another_index'),
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.8_sdk.xml> -
=========================== short test summary info ============================
FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
FAILED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
======= 3 failed, 1 passed, 5 skipped, 6 warnings in 2334.32s (0:38:54) ========
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221109-050736814268509
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:20f315811c205ed6157c7511f23e081061601dc8b0843b93da64a54acff22b7d
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:e39c94d2-01e6-46e4-8aca-1cf030930531
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:6605fac03404b1d02528d27d5428a7610e5b67abe5cabb88c14827c0f69a6001
Deleted: sha256:e6adfe5af0ef760dedccdee176c2ed46e93a19b3050731d34bb2c922ec56a70d
Deleted: sha256:1e7bb78e6108bbad953d5420186d217b31b9b7bdbc09ebdd0b66fa8b10fece0a
Deleted: sha256:d2bf0b40b56b6cd44a2dae1f10168760951255f74d57c5c4d7c7de41e1277542
Deleted: sha256:9576bb93737cb8cf165b72dd1b01c836ad179bf418c3de0802740b7135e29739
Deleted: sha256:d3bf5b93e1a87f8fad08d11c9402a4dc2c857538b8bfb5fcae0e25fcf79a357e
Deleted: sha256:b0efcceb718ed2c4941075a5a93591f355f5a1d382bb34829e9eb2272959c43a
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:20f315811c205ed6157c7511f23e081061601dc8b0843b93da64a54acff22b7d
Associated tags:
- 20221109-050736814268509
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221109-050736814268509
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221109-050736814268509].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:20f315811c205ed6157c7511f23e081061601dc8b0843b93da64a54acff22b7d].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:6605fac03404b1d02528d27d5428a7610e5b67abe5cabb88c14827c0f69a6001
Associated tags:
- e39c94d2-01e6-46e4-8aca-1cf030930531
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:e39c94d2-01e6-46e4-8aca-1cf030930531].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:6605fac03404b1d02528d27d5428a7610e5b67abe5cabb88c14827c0f69a6001].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9e486b9b79564f68c83b2e5b06aea3b3346617e7a64d0331af02cba37393f1c0
Associated tags:
- 8dd88dea-28ca-4789-9f71-e6942257f979
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:8dd88dea-28ca-4789-9f71-e6942257f979].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:9e486b9b79564f68c83b2e5b06aea3b3346617e7a64d0331af02cba37393f1c0].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py310:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 3m 38s
62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/f67hno3zczcek
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #9915
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9915/display/redirect?page=changes>
Changes:
[noreply] Update release notes. (#23986)
[noreply] [Go] Pipeline Resource Hints (#23990)
[noreply] [#21250] Trivial removal of loop over something that always has one
[noreply] Bump cloud.google.com/go/bigtable from 1.16.0 to 1.17.0 in /sdks
[noreply] Editorial review of the ML base API descriptions (#24026)
[noreply] Update my Twitter handle (#23653)
[noreply] Retroactively announce Batched DoFn support in 2.42.0 Blog (#24011)
[noreply] Bump cloud.google.com/go/storage from 1.27.0 to 1.28.0 in /sdks (#24028)
------------------------------------------
[...truncated 1.85 MB...]
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=1800 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)")
do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=1800 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:fc175c42d607351ebd87f8cb51d9ffde7846e797d1929a380a50c2743b9420e1
Associated tags:
- 20221108-225441798205716
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221108-225441798205716
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221108-225441798205716].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:fc175c42d607351ebd87f8cb51d9ffde7846e797d1929a380a50c2743b9420e1].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:bca6647296397310e0bc6fa43a9dfc2875b1eb26658589792e5727041e819249
Associated tags:
- c0006708-3c35-4a6b-991e-097bd9657bcd
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:c0006708-3c35-4a6b-991e-097bd9657bcd].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:bca6647296397310e0bc6fa43a9dfc2875b1eb26658589792e5727041e819249].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:264176e422e8feb9f6851bf364d2f30ac93d50947e39a9cc94f423c7ed587ff6
Associated tags:
- ab101931-bfe7-4151-b076-0bb3b8f4a1c5
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:ab101931-bfe7-4151-b076-0bb3b8f4a1c5].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:264176e422e8feb9f6851bf364d2f30ac93d50947e39a9cc94f423c7ed587ff6].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:740115e8853043ef0d0beea15161f9792b4361148390f9d86e4c52f56092b3d5
Associated tags:
- 20221108-225441798200485
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221108-225441798200485
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221108-225441798200485].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:740115e8853043ef0d0beea15161f9792b4361148390f9d86e4c52f56092b3d5].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:8822f2d971b66e7a787e89a513e251587b217481435095f2802852dcea1a9086
Associated tags:
- 7f0cdf47-0fdb-4dfc-9a76-38ebb8c3be8b
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:7f0cdf47-0fdb-4dfc-9a76-38ebb8c3be8b].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:8822f2d971b66e7a787e89a513e251587b217481435095f2802852dcea1a9086].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:e57a51b171a57cd1a41e80abba59275332075a321d46d6625db1f4b5d0634284
Associated tags:
- e54f8934-0551-47a9-97b6-ac73879b71ba
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:e54f8934-0551-47a9-97b6-ac73879b71ba].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:e57a51b171a57cd1a41e80abba59275332075a321d46d6625db1f4b5d0634284].
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:14b01ad4e7646bd906c764524531dbbd5766af16c38c9c20e0905978b753e90a
Associated tags:
- 20221108-225441806606004
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221108-225441806606004
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221108-225441806606004].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:14b01ad4e7646bd906c764524531dbbd5766af16c38c9c20e0905978b753e90a].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:e3ab4b6c76900f426b03f69d1c1a06bbbb834c6af07b170e9aedd20666cd4546
Associated tags:
- 418d91d9-e6b0-4e54-86f9-654e338ea4a5
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:418d91d9-e6b0-4e54-86f9-654e338ea4a5].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:e3ab4b6c76900f426b03f69d1c1a06bbbb834c6af07b170e9aedd20666cd4546].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:101a161833aa0cb98b5823bd651d153adbb0b9b2302c49a5fecc7033d505f098
Associated tags:
- 6f57a650-e6f6-4e9e-9005-09ef1dc86b99
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:6f57a650-e6f6-4e9e-9005-09ef1dc86b99].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:101a161833aa0cb98b5823bd651d153adbb0b9b2302c49a5fecc7033d505f098].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:f722bff14bfda2442dedbee353a33f21d680d3dba54ed92feb9934970bf0310d
Associated tags:
- 20221108-225441798200396
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221108-225441798200396
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221108-225441798200396].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:f722bff14bfda2442dedbee353a33f21d680d3dba54ed92feb9934970bf0310d].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:9f7e1ea41802e6e9b844ac14256f38a2a6a9dee8b63b2896594e5c548fe0a5c8
Associated tags:
- 7f0c5811-ee37-40d8-90cf-ee7e8a53388e
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:7f0c5811-ee37-40d8-90cf-ee7e8a53388e].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:9f7e1ea41802e6e9b844ac14256f38a2a6a9dee8b63b2896594e5c548fe0a5c8].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:6f9747addd73f8c924f617f7b0bc968473f5b41fa40a9a2973afb3f65e01fd96
Associated tags:
- 3c47d2da-314b-49b0-8f87-cab4687491dd
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:3c47d2da-314b-49b0-8f87-cab4687491dd].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:6f9747addd73f8c924f617f7b0bc968473f5b41fa40a9a2973afb3f65e01fd96].
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py310:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 55m 32s
62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/rdqlycsp5lghg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org