You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/08/30 17:25:58 UTC
Build failed in Jenkins: beam_PostCommit_Python_Examples_Spark #814
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/814/display/redirect?page=changes>
Changes:
[noreply] Fix yaml duplicated mapping key (#22952)
------------------------------------------
[...truncated 60.06 KB...]
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.40-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.1-py2.py3-none-any.whl (42 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Collecting cachetools<5,>=3.1.0
Using cached cachetools-4.2.4-py3-none-any.whl (10 kB)
Collecting google-apitools<0.5.32,>=0.5.31
Using cached google_apitools-0.5.31-py3-none-any.whl
Collecting google-api-core!=2.8.2,<3
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-2.11.0-py2.py3-none-any.whl (167 kB)
Collecting google-auth-httplib2<0.2.0,>=0.1.0
Using cached google_auth_httplib2-0.1.0-py2.py3-none-any.whl (9.3 kB)
Collecting google-cloud-datastore<2,>=1.8.0
Using cached google_cloud_datastore-1.15.5-py2.py3-none-any.whl (134 kB)
Collecting google-cloud-pubsub<3,>=2.1.0
Using cached google_cloud_pubsub-2.13.6-py2.py3-none-any.whl (235 kB)
Collecting google-cloud-pubsublite<2,>=1.2.0
Using cached google_cloud_pubsublite-1.4.3-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-bigquery<3,>=1.6.0
Using cached google_cloud_bigquery-2.34.4-py2.py3-none-any.whl (206 kB)
Collecting google-cloud-bigquery-storage<2.14,>=2.6.3
Using cached google_cloud_bigquery_storage-2.13.2-py2.py3-none-any.whl (180 kB)
Collecting google-cloud-core<3,>=0.28.1
Using cached google_cloud_core-2.3.2-py2.py3-none-any.whl (29 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.2-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.3-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.8.1-py2.py3-none-any.whl (119 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.2-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.1-py2.py3-none-any.whl (148 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.1-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.25.0-py3-none-any.whl (178 kB)
Collecting boto3>=1.9
Using cached boto3-1.24.62-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting botocore<1.28.0,>=1.27.62
Using cached botocore-1.27.62-py3-none-any.whl (9.1 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.2.0-py3-none-any.whl (17 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.10.0-py2.py3-none-any.whl (112 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.3.2
Using cached scipy-1.9.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.9 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (154 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)
Collecting docker>=4.0.0
Using cached docker-6.0.0-py3-none-any.whl (147 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.4.0-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (36 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2842755 sha256=253b2445a7205fd54ff577abea7ccadfb5964c714220aa834b9f6d02281789de
Stored in directory: /home/jenkins/.cache/pip/wheels/14/a0/51/7589d1703424aa51438d953c5a588a1947209e804f2fe0f996
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, tomli, threadpoolctl, tenacity, rsa, regex, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, overrides, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pytest, pyarrow, pandas, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest-timeout, pytest-forked, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-xdist, msrest, google-cloud-core, boto3, apache-beam, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
> Task :sdks:python:test-suites:portable:py37:installGcpTest
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.62 botocore-1.27.62 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.1 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-6.0.0 docopt-0.6.2 execnet-1.9.0 fastavro-1.6.0 fasteners-0.17.3 freezegun-1.2.2 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.11.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.6 google-cloud-pubsublite-1.4.3 google-cloud-recommendations-ai-0.7.1 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.3 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.8.0 overrides-6.2.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.10.0 proto-plus-1.22.1 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.2.1 pyyaml-6.0 regex-2022.8.17 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.10.0 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.40 tenacity-5.1.5 testcontainers-3.6.1 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.12 websocket-client-1.4.0 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:test-suites:portable:py39:installGcpTest
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.62 botocore-1.27.62 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.1 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-6.0.0 docopt-0.6.2 execnet-1.9.0 fastavro-1.6.0 fasteners-0.17.3 freezegun-1.2.2 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.11.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.6 google-cloud-pubsublite-1.4.3 google-cloud-recommendations-ai-0.7.1 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.3 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.22.4 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.8.0 overrides-6.2.0 pandas-1.4.3 parameterized-0.8.1 pbr-5.10.0 proto-plus-1.22.1 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.2.1 pyyaml-6.0 regex-2022.8.17 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.10.0 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.1.2 scipy-1.9.1 sqlalchemy-1.4.40 tenacity-5.1.5 testcontainers-3.6.1 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.12 websocket-client-1.4.0 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:test-suites:portable:py37:sparkExamples
>>> RUNNING integration tests with pipeline options: --runner=SparkRunner --project=apache-beam-testing --environment_type=LOOPBACK --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.42.0-SNAPSHOT.jar>
>>> pytest options: --log-cli-level=INFO
>>> collect markers: -m=examples_postcommit and not sickbay_spark
============================= test session starts ==============================
platform linux -- Python 3.7.12, pytest-7.1.2, pluggy-1.0.0
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python,> configfile: pytest.ini
plugins: xdist-2.5.0, timeout-2.1.0, forked-1.4.0, requests-mock-1.10.0
timeout: 600.0s
timeout method: signal
timeout func_only: False
----------------------------- live log collection ------------------------------
WARNING root:avroio_test.py:54 python-snappy is not installed; some tests will be skipped.
WARNING root:tfrecordio_test.py:55 Tensorflow is not installed, so skipping some tests.
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:191 Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:200 You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
collected 6553 items / 6530 deselected / 2 skipped / 23 selected
apache_beam/examples/wordcount_debugging_test.py::WordCountDebuggingTest::test_basics
-------------------------------- live log call ---------------------------------
INFO root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
> Task :sdks:python:test-suites:portable:py39:sparkExamples
>>> RUNNING integration tests with pipeline options: --runner=SparkRunner --project=apache-beam-testing --environment_type=LOOPBACK --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.42.0-SNAPSHOT.jar>
>>> pytest options: --log-cli-level=INFO
>>> collect markers: -m=examples_postcommit and not sickbay_spark
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-7.1.2, pluggy-1.0.0
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python,> configfile: pytest.ini
plugins: xdist-2.5.0, timeout-2.1.0, forked-1.4.0, requests-mock-1.10.0
timeout: 600.0s
timeout method: signal
timeout func_only: False
----------------------------- live log collection ------------------------------
WARNING root:avroio_test.py:54 python-snappy is not installed; some tests will be skipped.
WARNING root:tfrecordio_test.py:55 Tensorflow is not installed, so skipping some tests.
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:190 Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:199 You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.9_sdk:2.42.0.dev
collected 6553 items / 6530 deselected / 2 skipped / 23 selected
apache_beam/examples/wordcount_debugging_test.py::WordCountDebuggingTest::test_basics
-------------------------------- live log call ---------------------------------
INFO root:pipeline.py:185 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
Terminated
> Task :sdks:python:test-suites:portable:py39:sparkExamples FAILED
> Task :sdks:python:test-suites:portable:py37:sparkExamples FAILED
Terminated
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=421755e4-7ab0-4c4d-a751-a7fa0e09fc5d, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 3511700
log file: /home/jenkins/.gradle/daemon/7.4/daemon-3511700.out.log
----- Last 20 lines from daemon log file - daemon-3511700.out.log -----
WARNING root:tfrecordio_test.py:55 Tensorflow is not installed, so skipping some tests.
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:191 Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:200 You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:190 Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING apache_beam.runners.interactive.interactive_environment:interactive_environment.py:199 You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.9_sdk:2.42.0.dev
collected 6553 items / 6530 deselected / 2 skipped / 23 selected
apache_beam/examples/wordcount_debugging_test.py::WordCountDebuggingTest::test_basics
-------------------------------- live log call ---------------------------------
INFO root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
collected 6553 items / 6530 deselected / 2 skipped / 23 selected
apache_beam/examples/wordcount_debugging_test.py::WordCountDebuggingTest::test_basics
-------------------------------- live log call ---------------------------------
INFO root:pipeline.py:185 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
Terminated
Terminated
Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python_Examples_Spark #816
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/816/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_Examples_Spark #815
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/815/display/redirect?page=changes>
Changes:
[noreply] [Playground] [Infrastructure] Adding the Cloud Datastore client to save
[noreply] Fix jdbc date conversion offset 1 day (#22738)
[noreply] Set state integration test (#22935)
------------------------------------------
[...truncated 2.47 MB...]
top = '/tmp/tmp6sfp23g1', topdown = True, onerror = None, followlinks = False
def walk(top, topdown=True, onerror=None, followlinks=False):
"""Directory tree generator.
For each directory in the directory tree rooted at top (including top
itself, but excluding '.' and '..'), yields a 3-tuple
dirpath, dirnames, filenames
dirpath is a string, the path to the directory. dirnames is a list of
the names of the subdirectories in dirpath (excluding '.' and '..').
filenames is a list of the names of the non-directory files in dirpath.
Note that the names in the lists are just names, with no path components.
To get a full path (which begins with top) to a file or directory in
dirpath, do os.path.join(dirpath, name).
If optional arg 'topdown' is true or not specified, the triple for a
directory is generated before the triples for any of its subdirectories
(directories are generated top down). If topdown is false, the triple
for a directory is generated after the triples for all of its
subdirectories (directories are generated bottom up).
When topdown is true, the caller can modify the dirnames list in-place
(e.g., via del or slice assignment), and walk will only recurse into the
subdirectories whose names remain in dirnames; this can be used to prune the
search, or to impose a specific order of visiting. Modifying dirnames when
topdown is false has no effect on the behavior of os.walk(), since the
directories in dirnames have already been generated by the time dirnames
itself is generated. No matter the value of topdown, the list of
subdirectories is retrieved before the tuples for the directory and its
subdirectories are generated.
By default errors from the os.scandir() call are ignored. If
optional arg 'onerror' is specified, it should be a function; it
will be called with one argument, an OSError instance. It can
report the error to continue with the walk, or raise the exception
to abort the walk. Note that the filename is available as the
filename attribute of the exception object.
By default, os.walk does not follow symbolic links to subdirectories on
systems that support them. In order to get this functionality, set the
optional argument 'followlinks' to true.
Caution: if you pass a relative pathname for top, don't change the
current working directory between resumptions of walk. walk never
changes the current directory, and assumes that the client doesn't
either.
Example:
import os
from os.path import join, getsize
for root, dirs, files in os.walk('python/Lib/email'):
print(root, "consumes", end="")
print(sum([getsize(join(root, name)) for name in files]), end="")
print("bytes in", len(files), "non-directory files")
if 'CVS' in dirs:
dirs.remove('CVS') # don't visit CVS directories
"""
top = fspath(top)
dirs = []
nondirs = []
walk_dirs = []
# We may not have read permission for top, in which case we can't
# get a list of the files the directory contains. os.walk
# always suppressed the exception then, rather than blow up for a
# minor reason when (say) a thousand readable directories are still
# left to visit. That logic is copied here.
try:
# Note that scandir is global in this module due
# to earlier import-*.
scandir_it = scandir(top)
except OSError as error:
if onerror is not None:
onerror(error)
return
with scandir_it:
while True:
try:
try:
> entry = next(scandir_it)
E Failed: Timeout >600.0s
/usr/lib/python3.7/os.py:362: Failed
------------------------------ Captured log call -------------------------------
INFO root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
=============================== warnings summary ===============================
../../build/gradleenv/2022703441/lib/python3.7/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import load_source
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2285: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2291: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1129: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1131: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2774: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2776: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2807: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 671, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 654, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2281: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1661896279.862998419","description":"Error received from peer ipv6:[::1]:43517","file":"src/core/lib/surface/call.cc","file_line":966,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:47: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/dataframe/io.py>:637: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/fileio.py>:590: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/pytest_postCommitExamples-spark-py37.xml> -
=========================== short test summary info ============================
FAILED apache_beam/examples/wordcount_debugging_test.py::WordCountDebuggingTest::test_basics
= 1 failed, 21 passed, 3 skipped, 6530 deselected, 38 warnings in 1055.37s (0:17:35) =
> Task :sdks:python:test-suites:portable:py39:sparkExamples FAILED
> Task :sdks:python:test-suites:portable:py37:sparkExamples FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 271
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py39:sparkExamples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 271
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:sparkExamples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 20m 55s
80 actionable tasks: 55 executed, 23 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/tfwr6vahpg4ni
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org