You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2023/09/07 03:41:17 UTC
Build failed in Jenkins: beam_PostCommit_Python_Xlang_Gcp_Dataflow #652
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/652/display/redirect>
Changes:
------------------------------------------
[...truncated 644.15 KB...]
apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with
cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7149 deselected[0m, [33m76 warnings[0m[31m in 5554.89s (1:32:34)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageCleanup
Stopping expansion service pid: 2244940.
Skipping invalid pid: 2244941.
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T03:38:31.238Z: JOB_MESSAGE_BASIC: Finished operation Create/Impulse+Create/FlatMap(<lambda at core.py:3736>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T03:38:31.299Z: JOB_MESSAGE_BASIC: Executing operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T03:38:32.130Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T03:38:32.178Z: JOB_MESSAGE_BASIC: Executing operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Create/Map(decode)+WriteToBigTable/ParDo(_DirectRowMutationsToBeamRow)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/MapElements/Map/ParMultiDo(Anonymous)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigtableIO.Write/BigtableIO.WriteWithResults/ParDo(BigtableWriter)/ParMultiDo(BigtableWriter)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T03:39:00.449Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Create/Map(decode)+WriteToBigTable/ParDo(_DirectRowMutationsToBeamRow)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/MapElements/Map/ParMultiDo(Anonymous)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigtableIO.Write/BigtableIO.WriteWithResults/ParDo(BigtableWriter)/ParMultiDo(BigtableWriter)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T03:39:00.618Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T03:41:05.837Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:154 Job 2023-09-06_20_35_05-7232465567977362742 is in state JOB_STATE_DONE
[32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:190 Deleting table [test-table-1694057689-8ba698]
[32mPASSED[0m
[1m------------------------------ live log teardown -------------------------------[0m
[32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:198 Deleting instance [bt-write-xlang-1694056123-8dee47]
=================================== FAILURES ===================================
[31m[1m___________________ SnippetsTest.test_model_bigqueryio_xlang ___________________[0m
self = <apache_beam.examples.snippets.snippets_test.SnippetsTest testMethod=test_model_bigqueryio_xlang>
@pytest.mark.uses_gcp_java_expansion_service
@unittest.skipUnless(
os.environ.get('EXPANSION_PORT'),
"EXPANSION_PORT environment var is not provided.")
def test_model_bigqueryio_xlang(self):
p = TestPipeline()
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
> snippets.model_bigqueryio_xlang(p)
[1m[31mapache_beam/examples/snippets/snippets_test.py[0m:768:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/examples/snippets/snippets.py[0m:1065: in model_bigqueryio_xlang
quotes | "StorageWriteAPIWithFrequency" >> beam.io.WriteToBigQuery(
[1m[31mapache_beam/pvalue.py[0m:137: in __or__
return self.pipeline.apply(ptransform, self)
[1m[31mapache_beam/pipeline.py[0m:655: in apply
return self.apply(
[1m[31mapache_beam/pipeline.py[0m:666: in apply
return self.apply(transform, pvalueish)
[1m[31mapache_beam/pipeline.py[0m:712: in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:102: in apply
return super().apply(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:203: in apply
return self.apply_PTransform(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:207: in apply_PTransform
return transform.expand(input)
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2241: in expand
| "StorageWriteToBigQuery" >> StorageWriteToBigQuery(
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2540: in __init__
expansion_service or _default_io_expansion_service())
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2480: in _default_io_expansion_service
return BeamJarExpansionService(
[1m[31mapache_beam/transforms/external.py[0m:983: in __init__
path_to_jar = subprocess_server.JavaJarServer.path_to_beam_jar(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'apache_beam.utils.subprocess_server.JavaJarServer'>
gradle_target = 'sdks:java:io:google-cloud-platform:expansion-service:build'
appendix = None, version = '2.51.0.dev'
artifact_id = 'beam-sdks-java-io-google-cloud-platform-expansion-service'
@classmethod
def path_to_beam_jar(
cls,
gradle_target,
appendix=None,
version=beam_version,
artifact_id=None):
if gradle_target in cls._BEAM_SERVICES.replacements:
return cls._BEAM_SERVICES.replacements[gradle_target]
gradle_package = gradle_target.strip(':').rsplit(':', 1)[0]
if not artifact_id:
artifact_id = 'beam-' + gradle_package.replace(':', '-')
project_root = os.path.sep.join(
os.path.abspath(__file__).split(os.path.sep)[:-5])
local_path = os.path.join(
project_root,
gradle_package.replace(':', os.path.sep),
'build',
'libs',
cls.jar_name(
artifact_id,
version.replace('.dev', ''),
classifier='SNAPSHOT',
appendix=appendix))
if os.path.exists(local_path):
_LOGGER.info('Using pre-built snapshot at %s', local_path)
return local_path
elif '.dev' in version:
# TODO: Attempt to use nightly snapshots?
> raise RuntimeError(
(
'%s not found. '
'Please build the server with \n cd %s; ./gradlew %s') %
(local_path, os.path.abspath(project_root), gradle_target))
[1m[31mE RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with [0m
[1m[31mE cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build[0m
[1m[31mapache_beam/utils/subprocess_server.py[0m:243: RuntimeError
[33m=============================== warnings summary ===============================[0m
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py:28
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py>:28: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py:18
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py>:18: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: 19 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: 16 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2350: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py>:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/examples/snippets/snippets_test.py>:767: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2101: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2107: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with
cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7149 deselected[0m, [33m75 warnings[0m[31m in 5570.00s (1:32:50)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageCleanup
Stopping expansion service pid: 2245677.
Skipping invalid pid: 2245678.
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
Killing process at 2244319
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 42m 51s
119 actionable tasks: 83 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/eapde2i4wpuni
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python_Xlang_Gcp_Dataflow #656
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/656/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_Xlang_Gcp_Dataflow #655
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/655/display/redirect>
Changes:
------------------------------------------
[...truncated 667.34 KB...]
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with
cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7149 deselected[0m, [33m76 warnings[0m[31m in 5387.38s (1:29:47)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageCleanup
Stopping expansion service pid: 4107177.
Skipping invalid pid: 4107179.
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:53:20.742Z: JOB_MESSAGE_BASIC: All workers have finished the startup processes and began to receive work requests.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:53:22.935Z: JOB_MESSAGE_BASIC: Finished operation Create/Impulse+Create/FlatMap(<lambda at core.py:3736>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:53:22.977Z: JOB_MESSAGE_BASIC: Executing operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:53:24.191Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:53:24.238Z: JOB_MESSAGE_BASIC: Executing operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Create/Map(decode)+WriteToBigTable/ParDo(_DirectRowMutationsToBeamRow)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/MapElements/Map/ParMultiDo(Anonymous)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigtableIO.Write/BigtableIO.WriteWithResults/ParDo(BigtableWriter)/ParMultiDo(BigtableWriter)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:53:43.642Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Create/Map(decode)+WriteToBigTable/ParDo(_DirectRowMutationsToBeamRow)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/MapElements/Map/ParMultiDo(Anonymous)+WriteToBigTable/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigtableIO.Write/BigtableIO.WriteWithResults/ParDo(BigtableWriter)/ParMultiDo(BigtableWriter)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:53:43.828Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T21:56:05.306Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:154 Job 2023-09-07_14_49_59-7241551796547653941 is in state JOB_STATE_DONE
[32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:190 Deleting table [test-table-1694123385-3928f8]
[32mPASSED[0m
[1m------------------------------ live log teardown -------------------------------[0m
[32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:198 Deleting instance [bt-write-xlang-1694121933-005e1c]
=================================== FAILURES ===================================
[31m[1m___________________ SnippetsTest.test_model_bigqueryio_xlang ___________________[0m
self = <apache_beam.examples.snippets.snippets_test.SnippetsTest testMethod=test_model_bigqueryio_xlang>
@pytest.mark.uses_gcp_java_expansion_service
@unittest.skipUnless(
os.environ.get('EXPANSION_PORT'),
"EXPANSION_PORT environment var is not provided.")
def test_model_bigqueryio_xlang(self):
p = TestPipeline()
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
> snippets.model_bigqueryio_xlang(p)
[1m[31mapache_beam/examples/snippets/snippets_test.py[0m:768:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/examples/snippets/snippets.py[0m:1065: in model_bigqueryio_xlang
quotes | "StorageWriteAPIWithFrequency" >> beam.io.WriteToBigQuery(
[1m[31mapache_beam/pvalue.py[0m:137: in __or__
return self.pipeline.apply(ptransform, self)
[1m[31mapache_beam/pipeline.py[0m:655: in apply
return self.apply(
[1m[31mapache_beam/pipeline.py[0m:666: in apply
return self.apply(transform, pvalueish)
[1m[31mapache_beam/pipeline.py[0m:712: in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:102: in apply
return super().apply(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:203: in apply
return self.apply_PTransform(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:207: in apply_PTransform
return transform.expand(input)
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2241: in expand
| "StorageWriteToBigQuery" >> StorageWriteToBigQuery(
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2540: in __init__
expansion_service or _default_io_expansion_service())
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2480: in _default_io_expansion_service
return BeamJarExpansionService(
[1m[31mapache_beam/transforms/external.py[0m:983: in __init__
path_to_jar = subprocess_server.JavaJarServer.path_to_beam_jar(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'apache_beam.utils.subprocess_server.JavaJarServer'>
gradle_target = 'sdks:java:io:google-cloud-platform:expansion-service:build'
appendix = None, version = '2.51.0.dev'
artifact_id = 'beam-sdks-java-io-google-cloud-platform-expansion-service'
@classmethod
def path_to_beam_jar(
cls,
gradle_target,
appendix=None,
version=beam_version,
artifact_id=None):
if gradle_target in cls._BEAM_SERVICES.replacements:
return cls._BEAM_SERVICES.replacements[gradle_target]
gradle_package = gradle_target.strip(':').rsplit(':', 1)[0]
if not artifact_id:
artifact_id = 'beam-' + gradle_package.replace(':', '-')
project_root = os.path.sep.join(
os.path.abspath(__file__).split(os.path.sep)[:-5])
local_path = os.path.join(
project_root,
gradle_package.replace(':', os.path.sep),
'build',
'libs',
cls.jar_name(
artifact_id,
version.replace('.dev', ''),
classifier='SNAPSHOT',
appendix=appendix))
if os.path.exists(local_path):
_LOGGER.info('Using pre-built snapshot at %s', local_path)
return local_path
elif '.dev' in version:
# TODO: Attempt to use nightly snapshots?
> raise RuntimeError(
(
'%s not found. '
'Please build the server with \n cd %s; ./gradlew %s') %
(local_path, os.path.abspath(project_root), gradle_target))
[1m[31mE RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with [0m
[1m[31mE cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build[0m
[1m[31mapache_beam/utils/subprocess_server.py[0m:243: RuntimeError
[33m=============================== warnings summary ===============================[0m
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py:28
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/hdfs/config.py>:28: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py:18
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py>:18: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: 19 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: 16 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2350
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2350: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/google/rpc/__init__.py>:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/examples/snippets/snippets_test.py>:767: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2101: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2107: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with
cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7149 deselected[0m, [33m75 warnings[0m[31m in 5389.29s (1:29:49)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageCleanup
Stopping expansion service pid: 4117989.
Skipping invalid pid: 4117990.
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
Killing process at 4076109
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 44m 20s
119 actionable tasks: 83 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/xs4h6uogzlhte
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_Xlang_Gcp_Dataflow #654
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/654/display/redirect?page=changes>
Changes:
[noreply] adds maven to image (#28348)
------------------------------------------
[...truncated 668.21 KB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with
cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7149 deselected[0m, [33m75 warnings[0m[31m in 6064.53s (1:41:04)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageCleanup
Stopping expansion service pid: 2803492.
Skipping invalid pid: 2803493.
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:204 2023-09-07T15:49:48.032Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:154 Job 2023-09-07_08_43_22-9183506566346642196 is in state JOB_STATE_DONE
[32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:190 Deleting table [test-table-1694100947-bee7e4]
[32mPASSED[0m
[1m------------------------------ live log teardown -------------------------------[0m
[32mINFO [0m apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:198 Deleting instance [bt-write-xlang-1694099019-83ad22]
=================================== FAILURES ===================================
[31m[1m___________________ SnippetsTest.test_model_bigqueryio_xlang ___________________[0m
self = <apache_beam.examples.snippets.snippets_test.SnippetsTest testMethod=test_model_bigqueryio_xlang>
@pytest.mark.uses_gcp_java_expansion_service
@unittest.skipUnless(
os.environ.get('EXPANSION_PORT'),
"EXPANSION_PORT environment var is not provided.")
def test_model_bigqueryio_xlang(self):
p = TestPipeline()
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
> snippets.model_bigqueryio_xlang(p)
[1m[31mapache_beam/examples/snippets/snippets_test.py[0m:768:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/examples/snippets/snippets.py[0m:1065: in model_bigqueryio_xlang
quotes | "StorageWriteAPIWithFrequency" >> beam.io.WriteToBigQuery(
[1m[31mapache_beam/pvalue.py[0m:137: in __or__
return self.pipeline.apply(ptransform, self)
[1m[31mapache_beam/pipeline.py[0m:655: in apply
return self.apply(
[1m[31mapache_beam/pipeline.py[0m:666: in apply
return self.apply(transform, pvalueish)
[1m[31mapache_beam/pipeline.py[0m:712: in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:102: in apply
return super().apply(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:203: in apply
return self.apply_PTransform(transform, input, options)
[1m[31mapache_beam/runners/runner.py[0m:207: in apply_PTransform
return transform.expand(input)
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2241: in expand
| "StorageWriteToBigQuery" >> StorageWriteToBigQuery(
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2540: in __init__
expansion_service or _default_io_expansion_service())
[1m[31mapache_beam/io/gcp/bigquery.py[0m:2480: in _default_io_expansion_service
return BeamJarExpansionService(
[1m[31mapache_beam/transforms/external.py[0m:983: in __init__
path_to_jar = subprocess_server.JavaJarServer.path_to_beam_jar(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'apache_beam.utils.subprocess_server.JavaJarServer'>
gradle_target = 'sdks:java:io:google-cloud-platform:expansion-service:build'
appendix = None, version = '2.51.0.dev'
artifact_id = 'beam-sdks-java-io-google-cloud-platform-expansion-service'
@classmethod
def path_to_beam_jar(
cls,
gradle_target,
appendix=None,
version=beam_version,
artifact_id=None):
if gradle_target in cls._BEAM_SERVICES.replacements:
return cls._BEAM_SERVICES.replacements[gradle_target]
gradle_package = gradle_target.strip(':').rsplit(':', 1)[0]
if not artifact_id:
artifact_id = 'beam-' + gradle_package.replace(':', '-')
project_root = os.path.sep.join(
os.path.abspath(__file__).split(os.path.sep)[:-5])
local_path = os.path.join(
project_root,
gradle_package.replace(':', os.path.sep),
'build',
'libs',
cls.jar_name(
artifact_id,
version.replace('.dev', ''),
classifier='SNAPSHOT',
appendix=appendix))
if os.path.exists(local_path):
_LOGGER.info('Using pre-built snapshot at %s', local_path)
return local_path
elif '.dev' in version:
# TODO: Attempt to use nightly snapshots?
> raise RuntimeError(
(
'%s not found. '
'Please build the server with \n cd %s; ./gradlew %s') %
(local_path, os.path.abspath(project_root), gradle_target))
[1m[31mE RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with [0m
[1m[31mE cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build[0m
[1m[31mapache_beam/utils/subprocess_server.py[0m:243: RuntimeError
[33m=============================== warnings summary ===============================[0m
../../build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py:28
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/hdfs/config.py>:28: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py:18
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py>:18: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: 19 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: 16 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2350
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2350: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/pkg_resources/__init__.py>:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/rpc/__init__.py>:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import util
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/examples/snippets/snippets_test.py>:767: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location = 'gs://mylocation'
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2101: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/examples/snippets/snippets_test.py::SnippetsTest::test_model_bigqueryio_xlang
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_nested_records_and_lists
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_at_least_once
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_streaming_with_auto_sharding
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_with_at_least_once_semantics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2107: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/external/xlang_bigqueryio_it_test.py: 10 warnings
apache_beam/io/gcp/bigtableio_it_test.py: 6 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/apache_beam/transforms/external.py>:679: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self._expansion_service, pipeline.options)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/examples/snippets/snippets_test.py::[1mSnippetsTest::test_model_bigqueryio_xlang[0m - RuntimeError: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/java/io/google-cloud-platform/expansion-service/build/libs/beam-sdks-java-io-google-cloud-platform-expansion-service-2.51.0-SNAPSHOT.jar> not found. Please build the server with
cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src;> ./gradlew sdks:java:io:google-cloud-platform:expansion-service:build
[31m= [31m[1m1 failed[0m, [32m13 passed[0m, [33m17 skipped[0m, [33m7149 deselected[0m, [33m76 warnings[0m[31m in 6341.36s (1:45:41)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageCleanup
Stopping expansion service pid: 2802757.
Skipping invalid pid: 2802758.
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
Killing process at 2802146
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 51m 43s
119 actionable tasks: 83 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/2fei56be65dee
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_Xlang_Gcp_Dataflow #653
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/653/display/redirect>
Changes:
------------------------------------------
[...truncated 81.33 KB...]
Using cached greenlet-3.0.0rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached scipy-1.11.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (36.3 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3300692 sha256=586b67ad8ae02eec7adba1594bf938e0fb7b5854ebc505820cd6da911e7644eb
Stored in directory: /home/jenkins/.cache/pip/wheels/e4/5c/da/6329b44131b6caad3634eee5a8a0f000d8cb8a636243c55eec
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Attempting uninstall: numpy
Found existing installation: numpy 1.25.2
Uninstalling numpy-1.25.2:
Successfully uninstalled numpy-1.25.2
> Task :sdks:python:installGcpTest
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3300692 sha256=5a92cafdf804cc769c6bcf2a2d794a0835b99b5d042a724b3a8fd5c656ab2f33
Stored in directory: /home/jenkins/.cache/pip/wheels/e8/b9/af/c837b966c32f7e8c228f9347d52f4fecbafafcf20cf936a4d7
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.42 botocore-1.31.42 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.1 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:test-suites:dataflow:py38:installGcpTest
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.42 botocore-1.31.42 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.1 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:test-suites:xlang:fnApiJobServerSetup
> Task :sdks:python:bdistPy311linux
Unable to find image 'quay.io/pypa/manylinux2014_x86_64:2022-08-09-51a01be' locally
2022-08-09-51a01be: Pulling from pypa/manylinux2014_x86_64
2d473b07cdd5: Pulling fs layer
1754cc444d30: Pulling fs layer
05b4009d24a5: Pulling fs layer
b0e57067162d: Pulling fs layer
c83ea453718c: Pulling fs layer
72eadfe2866c: Pulling fs layer
d8c9e4ddbc49: Pulling fs layer
44638940d548: Pulling fs layer
d330e4bcae62: Pulling fs layer
840987cd400a: Pulling fs layer
ad8c1849b6d1: Pulling fs layer
63b9b98c85bf: Pulling fs layer
f24d4d16d872: Pulling fs layer
fc087ea09d79: Pulling fs layer
913d8dcfc27c: Pulling fs layer
b0e57067162d: Waiting
d8c9e4ddbc49: Waiting
937c8577b1bf: Pulling fs layer
c83ea453718c: Waiting
44638940d548: Waiting
3f4a4d73c524: Pulling fs layer
4c040383c802: Pulling fs layer
63b9b98c85bf: Waiting
561cecff1472: Pulling fs layer
72eadfe2866c: Waiting
f24d4d16d872: Waiting
b9a1894f19c6: Pulling fs layer
ad8c1849b6d1: Waiting
840987cd400a: Waiting
8a05d7fdaf76: Pulling fs layer
3f4a4d73c524: Waiting
215fba5e06c9: Pulling fs layer
4c040383c802: Waiting
fc087ea09d79: Waiting
937c8577b1bf: Waiting
b9a1894f19c6: Waiting
8a05d7fdaf76: Waiting
561cecff1472: Waiting
913d8dcfc27c: Waiting
215fba5e06c9: Waiting
1754cc444d30: Verifying Checksum
1754cc444d30: Download complete
05b4009d24a5: Verifying Checksum
05b4009d24a5: Download complete
b0e57067162d: Verifying Checksum
b0e57067162d: Download complete
c83ea453718c: Verifying Checksum
c83ea453718c: Download complete
72eadfe2866c: Download complete
d330e4bcae62: Verifying Checksum
d330e4bcae62: Download complete
2d473b07cdd5: Verifying Checksum
2d473b07cdd5: Download complete
ad8c1849b6d1: Verifying Checksum
ad8c1849b6d1: Download complete
840987cd400a: Verifying Checksum
840987cd400a: Download complete
f24d4d16d872: Download complete
63b9b98c85bf: Download complete
913d8dcfc27c: Download complete
fc087ea09d79: Verifying Checksum
fc087ea09d79: Download complete
3f4a4d73c524: Verifying Checksum
3f4a4d73c524: Download complete
d8c9e4ddbc49: Verifying Checksum
d8c9e4ddbc49: Download complete
937c8577b1bf: Download complete
b9a1894f19c6: Verifying Checksum
b9a1894f19c6: Download complete
8a05d7fdaf76: Download complete
4c040383c802: Download complete
2d473b07cdd5: Pull complete
215fba5e06c9: Verifying Checksum
215fba5e06c9: Download complete
1754cc444d30: Pull complete
05b4009d24a5: Pull complete
b0e57067162d: Pull complete
c83ea453718c: Pull complete
72eadfe2866c: Pull complete
561cecff1472: Verifying Checksum
561cecff1472: Download complete
d8c9e4ddbc49: Pull complete
44638940d548: Pull complete
d330e4bcae62: Pull complete
840987cd400a: Pull complete
ad8c1849b6d1: Pull complete
63b9b98c85bf: Pull complete
f24d4d16d872: Pull complete
fc087ea09d79: Pull complete
913d8dcfc27c: Pull complete
937c8577b1bf: Pull complete
3f4a4d73c524: Pull complete
4c040383c802: Pull complete
561cecff1472: Pull complete
b9a1894f19c6: Pull complete
8a05d7fdaf76: Pull complete
215fba5e06c9: Pull complete
Digest: sha256:d195b4906bd74dcc9db60e8ffa594dd37f5eb7e9ce41dc1683cf9458a22ddf2e
Status: Downloaded newer image for quay.io/pypa/manylinux2014_x86_64:2022-08-09-51a01be
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip available: 22.2.2 -> 23.2.1
[notice] To update, run: python -m pip install --upgrade pip
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip available: 22.2.2 -> 23.2.1
[notice] To update, run: python -m pip install --upgrade pip
[notice] A new release of pip available: 22.2.2 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
INFO:auditwheel.main_repair:Repairing apache_beam-2.51.0.dev0-cp311-cp311-linux_x86_64.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_x86_64, manylinux2014_x86_64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp311-cp311-linux_x86_64
INFO:auditwheel.wheeltools:New WHEEL info tags: cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-manylinux2014_x86_64
INFO:auditwheel.main_repair:
Fixed-up wheel written to /tmp/cibuildwheel/repaired_wheel/apache_beam-2.51.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
> Task :sdks:python:test-suites:dataflow:py311:initializeForDataflowJob
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageSetup
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/scripts/run_expansion_services.sh>: line 128: /bin/activate: No such file or directory
> Task :sdks:python:bdistPy38linux
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip available: 22.2.2 -> 23.2.1
[notice] To update, run: python -m pip install --upgrade pip
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip available: 22.2.2 -> 23.2.1
[notice] To update, run: python -m pip install --upgrade pip
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava
> Task :sdks:python:bdistPy38linux
[notice] A new release of pip available: 22.2.2 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
INFO:auditwheel.main_repair:Repairing apache_beam-2.51.0.dev0-cp38-cp38-linux_x86_64.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_x86_64, manylinux2014_x86_64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp38-cp38-linux_x86_64
INFO:auditwheel.wheeltools:New WHEEL info tags: cp38-cp38-manylinux_2_17_x86_64, cp38-cp38-manylinux2014_x86_64
INFO:auditwheel.main_repair:
Fixed-up wheel written to /tmp/cibuildwheel/repaired_wheel/apache_beam-2.51.0.dev0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
> Task :sdks:python:test-suites:dataflow:py38:initializeForDataflowJob
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageSetup
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/scripts/run_expansion_services.sh>: line 128: /bin/activate: No such file or directory
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageCleanup
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava FAILED
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageCleanup
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 40m 1s
119 actionable tasks: 83 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/7lzlium5kechy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org