You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/10/29 16:28:49 UTC
Build failed in Jenkins: beam_LoadTests_Python_coGBK_Flink_Batch
#96
See <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/96/display/redirect?page=changes>
Changes:
[sunjincheng121] [BEAM-8442] Remove duplicate code for bundle register in Python SDK
[robertwb] Move use_sdf_bounded_source tests to iobase.
[hsuryawirawan] Change from study_project.xml to yaml files for Beam Katas Python
[hsuryawirawan] Exclude Katas remote-info.yaml files from RAT check
[hsuryawirawan] Add note for the "Built-in IOs" tasks to inform that there is no kata to
[mxm] [BEAM-8383] Fix gauge metrics in Python SDK and Flink runner
[lukecwik] [BEAM-8432] Move javaVersion to gradle.properties (#9830)
[lostluck] Enables Dataflow workers to use private IPs
[pabloem] Update the Beam Katas Java to use the new course metadata yaml format
[lukecwik] BEAM-3658 Port SpannerIOReadTest off DoFnTester (#9898)
[mxm] [BEAM-8383] Add metrics for the Python state cache
[github] Increasing timeout due to still constant timeouts.
[thw] [BEAM-8471] Flink native job submission for portable pipelines
[tweise] [BEAM-8471] Put back FlinkJobServerDriver.fromParams to fix post commit
[kcweaver] [BEAM-8509] TestPortableRunner uses JobServerDriver interface
[lukasz.gajowy] [BEAM-6303] Add withSuffix(".parquet") to Parquet write javadoc.
[mxm] [BEAM-7730] Add Flink 1.9 to the list of supported Flink versions
------------------------------------------
[...truncated 99.17 KB...]
Processing /home/jenkins/.cache/pip/wheels/74/bc/50/ae030267a192919b289f84661cfeb5573cf383841c543e8696/future-0.18.1-cp27-none-any.whl
Requirement already satisfied: grpcio<2,>=1.12.1 in <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (1.24.3)
Processing /home/jenkins/.cache/pip/wheels/fe/a7/05/23e3699975fc20f8a30e00ac1e515ab8c61168e982abe4ce70/hdfs-2.5.8-cp27-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/6d/41/4b/2b369d6e2b7eaebcdd423516d3fb659c7658c16a2be8fd04ec/httplib2-0.12.0-cp27-none-any.whl
Collecting mock<3.0.0,>=1.0.1
Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting pymongo<4.0.0,>=3.8.0
Using cached https://files.pythonhosted.org/packages/00/5c/5379d5b8167a5938918d9ee147f865f6f8a64b93947d402cfdca5c1416d2/pymongo-3.9.0-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/48/f7/87/b932f09c6335dbcf45d916937105a372ab14f353a9ca431d7d/oauth2client-3.0.0-cp27-none-any.whl
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (3.10.0)
Collecting pydot<2,>=1.2.0
Using cached https://files.pythonhosted.org/packages/33/d1/b1479a770f66d962f545c2101630ce1d5592d90cb4f083d38862e93d16d2/pydot-1.4.1-py2.py3-none-any.whl
Collecting python-dateutil<3,>=2.8.0
Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting pytz>=2018.3
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/fe/03/0c/354f2a3b1e10ab337b45728410509de62fbc0f7fe09ad196a2/avro-1.9.1-cp27-none-any.whl
Collecting funcsigs<2,>=1.0.2
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: futures<4.0.0,>=3.2.0 in <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from apache-beam==2.18.0.dev0) (3.3.0)
Processing /home/jenkins/.cache/pip/wheels/81/91/41/3272543c0b9c61da9c525f24ee35bae6fe8f60d4858c66805d/PyVCF-0.6.8-cp27-none-any.whl
Collecting typing<3.7.0,>=3.6.0
Using cached https://files.pythonhosted.org/packages/cc/3e/29f92b7aeda5b078c86d14f550bf85cff809042e3429ace7af6193c3bc9f/typing-3.6.6-py2-none-any.whl
Collecting pyarrow<0.14.0,>=0.11.1
Using cached https://files.pythonhosted.org/packages/83/41/552ed611e2fe1ffca29df430a932426e6b5ce184a1fa26f5b954b0da4909/pyarrow-0.13.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting cachetools<4,>=3.1.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting google-apitools<0.5.29,>=0.5.28
Using cached https://files.pythonhosted.org/packages/07/5e/3e04cb66f5ced9267a854184bb09863d85d199646ea8480fee26b4313a00/google_apitools-0.5.28-py2-none-any.whl
Collecting google-cloud-datastore<1.8.0,>=1.7.1
Using cached https://files.pythonhosted.org/packages/d0/aa/29cbcf8cf7d08ce2d55b9dce858f7c632b434cb6451bed17cb4275804217/google_cloud_datastore-1.7.4-py2.py3-none-any.whl
Collecting google-cloud-pubsub<1.1.0,>=0.39.0
Using cached https://files.pythonhosted.org/packages/d3/91/07a82945a7396ea34debafd476724bb5fc267c292790fdf2138c693f95c5/google_cloud_pubsub-1.0.2-py2.py3-none-any.whl
Collecting google-cloud-bigquery<1.18.0,>=1.6.0
Using cached https://files.pythonhosted.org/packages/a4/96/1b9cf1d43869c47a205aad411dac7c3040df6093d63c39273fa4d4c45da7/google_cloud_bigquery-1.17.1-py2.py3-none-any.whl
Collecting google-cloud-core<2,>=0.28.1
Using cached https://files.pythonhosted.org/packages/ee/f0/084f598629db8e6ec3627688723875cdb03637acb6d86999bb105a71df64/google_cloud_core-1.0.3-py2.py3-none-any.whl
Collecting google-cloud-bigtable<1.1.0,>=0.31.1
Using cached https://files.pythonhosted.org/packages/95/af/0ef7d097a1d5ad0c843867600e86de915e8ab8864740f49a4636cfb51af6/google_cloud_bigtable-1.0.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/bd/ce/33/8b769968db3761c42c7a91d8a0dbbafc50acfa0750866c8abd/proto_google_cloud_datastore_v1-0.90.4-cp27-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/09/61/a5/7e8f4442b3c3d406ee9eb6c06e1ecbe5625f62f8cb19c08f5b/googledatastore-7.0.2-cp27-none-any.whl
Collecting nose>=1.3.7
Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/c4/1f/cd/9250fbf2fcc179e28bb4f7ee26a4fc7525914469d83a4f0c09/nose_xunitmp-0.4.1-cp27-none-any.whl
Collecting numpy<2,>=1.14.3
Using cached https://files.pythonhosted.org/packages/d7/b1/3367ea1f372957f97a6752ec725b87886e12af1415216feec9067e31df70/numpy-1.16.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pandas<0.25,>=0.23.4
Using cached https://files.pythonhosted.org/packages/db/83/7d4008ffc2988066ff37f6a0bb6d7b60822367dcb36ba5e39aa7801fda54/pandas-0.24.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting parameterized<0.7.0,>=0.6.0
Using cached https://files.pythonhosted.org/packages/3a/49/75f6dadb09e2f8ace3cdffe0c99a04f1b98dff41fbf9e768665d8b469e29/parameterized-0.6.3-py2.py3-none-any.whl
Collecting pyhamcrest<2.0,>=1.9
Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030/PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl
Collecting requests_mock<2.0,>=1.7
Using cached https://files.pythonhosted.org/packages/8c/f1/66c54a412543b29454102ae74b1454fce2d307b1c36e6bd2e9818394df88/requests_mock-1.7.0-py2.py3-none-any.whl
Collecting tenacity<6.0,>=5.0.2
Using cached https://files.pythonhosted.org/packages/c3/c2/e0cbe473037b8826fef99f9d52146647cab24f53586b524d732521bf0f49/tenacity-5.1.4-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.12.1->apache-beam==2.18.0.dev0) (1.12.0)
Requirement already satisfied: enum34>=1.0.4; python_version < "3.4" in <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from grpcio<2,>=1.12.1->apache-beam==2.18.0.dev0) (1.1.6)
Processing /home/jenkins/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e/docopt-0.6.2-py2.py3-none-any.whl
Collecting requests>=2.7.0
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting pbr>=0.11
Using cached https://files.pythonhosted.org/packages/46/a4/d5c83831a3452713e4b4f126149bc4fbda170f7cb16a86a00ce57ce0e9ad/pbr-5.4.3-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7
Using cached https://files.pythonhosted.org/packages/a1/71/8f0d444e3a74e5640a3d5d967c1c6b015da9c655f35b2d308a55d907a517/pyasn1-0.4.7-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Collecting rsa>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages> (from protobuf<4,>=3.5.0.post1->apache-beam==2.18.0.dev0) (41.6.0)
Collecting pyparsing>=2.1.4
Using cached https://files.pythonhosted.org/packages/11/fa/0160cd525c62d7abd076a070ff02b2b94de589f1a9789774f17d7c54058e/pyparsing-2.4.2-py2.py3-none-any.whl
Collecting fasteners>=0.14
Using cached https://files.pythonhosted.org/packages/18/bd/55eb2d6397b9c0e263af9d091ebdb756b15756029b3cededf6461481bc63/fasteners-0.15-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.6.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/de/3a/83/77a1e18e1a8757186df834b86ce6800120ac9c79cd8ca4091b/grpc_google_iam_v1-0.12.3-cp27-none-any.whl
Collecting google-resumable-media<0.5.0dev,>=0.3.1
Using cached https://files.pythonhosted.org/packages/96/d7/b29a41b01b854480891dfc408211ffb0cc7a2a3d5f15a3b6740ec18c845b/google_resumable_media-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting monotonic>=0.6; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/e0/da/55f51ea951e1b7c63a579c09dd7db825bb730ec1fe9c0180fc77bfb31448/urllib3-1.25.6-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/c5/9b/ed0516cc1f7609fb0217e3057ff4f0f9f3e3ce79a369c6af4a6c5ca25664/google_auth-1.6.3-py2.py3-none-any.whl
Installing collected packages: crcmod, dill, fastavro, future, docopt, urllib3, certifi, chardet, idna, requests, hdfs, httplib2, pbr, funcsigs, mock, pymongo, pyasn1, pyasn1-modules, rsa, oauth2client, pyparsing, pydot, python-dateutil, pytz, avro, pyvcf, typing, numpy, pyarrow, cachetools, monotonic, fasteners, google-apitools, googleapis-common-protos, google-auth, google-api-core, google-cloud-core, google-cloud-datastore, grpc-google-iam-v1, google-cloud-pubsub, google-resumable-media, google-cloud-bigquery, google-cloud-bigtable, proto-google-cloud-datastore-v1, googledatastore, nose, nose-xunitmp, pandas, parameterized, pyhamcrest, pyyaml, requests-mock, tenacity, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam avro-1.9.1 cachetools-3.1.1 certifi-2019.9.11 chardet-3.0.4 crcmod-1.7 dill-0.3.0 docopt-0.6.2 fastavro-0.21.24 fasteners-0.15 funcsigs-1.0.2 future-0.18.1 google-api-core-1.14.3 google-apitools-0.5.28 google-auth-1.6.3 google-cloud-bigquery-1.17.1 google-cloud-bigtable-1.0.0 google-cloud-core-1.0.3 google-cloud-datastore-1.7.4 google-cloud-pubsub-1.0.2 google-resumable-media-0.4.1 googleapis-common-protos-1.6.0 googledatastore-7.0.2 grpc-google-iam-v1-0.12.3 hdfs-2.5.8 httplib2-0.12.0 idna-2.8 mock-2.0.0 monotonic-1.5 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.16.5 oauth2client-3.0.0 pandas-0.24.2 parameterized-0.6.3 pbr-5.4.3 proto-google-cloud-datastore-v1-0.90.4 pyarrow-0.13.0 pyasn1-0.4.7 pyasn1-modules-0.2.7 pydot-1.4.1 pyhamcrest-1.9.0 pymongo-3.9.0 pyparsing-2.4.2 python-dateutil-2.8.0 pytz-2019.3 pyvcf-0.6.8 pyyaml-5.1.2 requests-2.22.0 requests-mock-1.7.0 rsa-4.0 tenacity-5.1.4 typing-3.6.6 urllib3-1.25.6
> Task :sdks:python:apache_beam:testing:load_tests:run
setup.py:191: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/build/gradleenv/1329484227/local/lib/python2.7/site-packages/setuptools/dist.py>:475: UserWarning: Normalizing '2.18.0.dev' to '2.18.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/sdks/python/apache_beam/__init__.py>:84: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
testCoGroupByKey (apache_beam.testing.load_tests.co_group_by_key_test.CoGroupByKeyTest) ... ERROR
======================================================================
ERROR: testCoGroupByKey (apache_beam.testing.load_tests.co_group_by_key_test.CoGroupByKeyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 70, in tearDown
result = self.pipeline.run()
File "<https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/test_pipeline.py",> line 114, in run
state = result.wait_until_finish()
File "<https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 464, in wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline load-tests-python-flink-batch-cogbk-1-1029150511_caef9767-f7a5-4455-baed-8ed356d79238 failed in state FAILED: java.io.IOException: Received exit code 126 for command 'docker run -d --network=host --env=DOCKER_MAC_CONTAINER=null --rm gcr.io/apache-beam-testing/beam_portability/python2.7_sdk:latest --id=1-1 --logging_endpoint=localhost:46115 --artifact_endpoint=localhost:33615 --provision_endpoint=localhost:38495 --control_endpoint=localhost:41089'. stderr: docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create: dial unix /var/run/docker.sock: connect: permission denied.See 'docker run --help'.
-------------------- >> begin captured logging << --------------------
root: INFO: Metrics will not be collected
root: INFO: ==================== <function lift_combiners at 0x7f90fdee5e60> ====================
root: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
root: DEBUG: Stages: ['ref_AppliedPTransform_Read pc1/Impulse_3\n Read pc1/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Split_4\n Read pc1/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/AddRandomKeys_6\n Read pc1/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read pc1/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read pc1/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read pc1/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/RemoveRandomKeys_14\n Read pc1/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/ReadSplits_15\n Read pc1/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Make pc1 iterable_16\n Make pc1 iterable:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start pc1_17\n Measure time: Start pc1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Impulse_19\n Read pc2/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Split_20\n Read pc2/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/AddRandomKeys_22\n Read pc2/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n Read pc2/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/ReshufflePerKey/GroupByKey_25\n Read pc2/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n Read pc2/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/RemoveRandomKeys_30\n Read pc2/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/ReadSplits_31\n Read pc2/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Make pc2 iterable_32\n Make pc2 iterable:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start pc2_33\n Measure time: Start pc2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /pair_with_pc2_35\n CoGroupByKey /pair_with_pc2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /pair_with_pc1_36\n CoGroupByKey /pair_with_pc1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /Flatten_37\n CoGroupByKey /Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /GroupByKey_38\n CoGroupByKey /GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /Map(_merge_tagged_vals_under_key)_42\n CoGroupByKey /Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume Joined Collections_43\n Consume Joined Collections:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End_44\n Measure time: End:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
root: INFO: ==================== <function expand_sdf at 0x7f90fdee5ed8> ====================
root: DEBUG: 27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
root: DEBUG: Stages: ['ref_AppliedPTransform_Read pc1/Impulse_3\n Read pc1/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Split_4\n Read pc1/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/AddRandomKeys_6\n Read pc1/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_8\n Read pc1/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/ReshufflePerKey/GroupByKey_9\n Read pc1/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_13\n Read pc1/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/Reshuffle/RemoveRandomKeys_14\n Read pc1/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc1/ReadSplits_15\n Read pc1/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Make pc1 iterable_16\n Make pc1 iterable:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start pc1_17\n Measure time: Start pc1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Impulse_19\n Read pc2/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Split_20\n Read pc2/Split:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/AddRandomKeys_22\n Read pc2/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/ReshufflePerKey/Map(reify_timestamps)_24\n Read pc2/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/ReshufflePerKey/GroupByKey_25\n Read pc2/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)_29\n Read pc2/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/Reshuffle/RemoveRandomKeys_30\n Read pc2/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Read pc2/ReadSplits_31\n Read pc2/ReadSplits:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Make pc2 iterable_32\n Make pc2 iterable:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: Start pc2_33\n Measure time: Start pc2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /pair_with_pc2_35\n CoGroupByKey /pair_with_pc2:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /pair_with_pc1_36\n CoGroupByKey /pair_with_pc1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /Flatten_37\n CoGroupByKey /Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /GroupByKey_38\n CoGroupByKey /GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_CoGroupByKey /Map(_merge_tagged_vals_under_key)_42\n CoGroupByKey /Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Consume Joined Collections_43\n Consume Joined Collections:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Measure time: End_44\n Measure time: End:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>']
root: DEBUG: Runner option 'job_name' was already added
root: DEBUG: Runner option 'runner' was already added
root: DEBUG: Runner option 'temp_location' was already added
root: DEBUG: Runner option 'streaming' was already added
root: DEBUG: Runner option 'environment_cache_millis' was already added
root: DEBUG: Runner option 'files_to_stage' was already added
root: DEBUG: Runner option 'job_endpoint' was already added
root: DEBUG: Runner option 'sdk_worker_parallelism' was already added
root: DEBUG: Runner option 'dataflow_kms_key' was already added
root: DEBUG: Runner option 'enable_streaming_engine' was already added
root: DEBUG: Runner option 'project' was already added
root: DEBUG: Runner option 'zone' was already added
root: DEBUG: Runner option 'experiments' was already added
root: DEBUG: Runner option 'pubsub_root_url' was already added
root: WARNING: Discarding unparseable args: ['--publish_to_big_query=false', '--metrics_dataset=load_test', '--metrics_table=python_flink_batch_cogbk_1', '--input_options={"num_records": 20000000,"key_size": 10,"value_size": 90,"num_hot_keys": 1,"hot_key_fraction": 1}', '--co_input_options={"num_records": 20000000,"key_size": 10,"value_size": 90,"num_hot_keys": 1,"hot_key_fraction": 1}', '--iterations=1']
root: INFO: Job state changed to RUNNING
root: DEBUG: org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 2ebcc60e36ca2226c3525bf3698756dd)
at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:301)
at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:209)
at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:186)
at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:173)
at org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator$BatchTranslationContext.execute(FlinkBatchPortablePipelineTranslator.java:191)
at org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:105)
at org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:81)
at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:78)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259)
... 16 more
Caused by: java.lang.Exception: The user defined 'open()' method caused an exception: java.io.IOException: Received exit code 126 for command 'docker run -d --network=host --env=DOCKER_MAC_CONTAINER=null --rm gcr.io/apache-beam-testing/beam_portability/python2.7_sdk:latest --id=1-1 --logging_endpoint=localhost:46115 --artifact_endpoint=localhost:33615 --provision_endpoint=localhost:38495 --control_endpoint=localhost:41089'. stderr: docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create: dial unix /var/run/docker.sock: connect: permission denied.See 'docker run --help'.
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:499)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
... 1 more
Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Received exit code 126 for command 'docker run -d --network=host --env=DOCKER_MAC_CONTAINER=null --rm gcr.io/apache-beam-testing/beam_portability/python2.7_sdk:latest --id=1-1 --logging_endpoint=localhost:46115 --artifact_endpoint=localhost:33615 --provision_endpoint=localhost:38495 --control_endpoint=localhost:41089'. stderr: docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create: dial unix /var/run/docker.sock: connect: permission denied.See 'docker run --help'.
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4966)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:214)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:205)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:188)
at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.getStageBundleFactory(DefaultExecutableStageContext.java:42)
at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingExecutableStageContextFactory.java:198)
at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.open(FlinkExecutableStageFunction.java:135)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:495)
... 4 more
Caused by: java.io.IOException: Received exit code 126 for command 'docker run -d --network=host --env=DOCKER_MAC_CONTAINER=null --rm gcr.io/apache-beam-testing/beam_portability/python2.7_sdk:latest --id=1-1 --logging_endpoint=localhost:46115 --artifact_endpoint=localhost:33615 --provision_endpoint=localhost:38495 --control_endpoint=localhost:41089'. stderr: docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create: dial unix /var/run/docker.sock: connect: permission denied.See 'docker run --help'.
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at org.apache.beam.runners.fnexecution.environment.DockerCommand.runImage(DockerCommand.java:92)
at org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:159)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:181)
at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:165)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
... 12 more
root: ERROR: java.io.IOException: Received exit code 126 for command 'docker run -d --network=host --env=DOCKER_MAC_CONTAINER=null --rm gcr.io/apache-beam-testing/beam_portability/python2.7_sdk:latest --id=1-1 --logging_endpoint=localhost:46115 --artifact_endpoint=localhost:33615 --provision_endpoint=localhost:38495 --control_endpoint=localhost:41089'. stderr: docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create: dial unix /var/run/docker.sock: connect: permission denied.See 'docker run --help'.
root: INFO: Job state changed to FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 1 test in 26.202s
FAILED (errors=1)
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 55
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 55s
3 actionable tasks: 3 executed
Publishing build scan...
https://gradle.com/s/3e32hcsewakkq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_LoadTests_Python_coGBK_Flink_Batch #97
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_LoadTests_Python_coGBK_Flink_Batch/97/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org