You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/11/20 01:28:47 UTC
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#756
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/756/display/redirect?page=changes>
Changes:
[markliu] Add Java mobile game on DirectRunner to release script
[suztomo] empty commit
[suztomo] Revert empty line
[suztomo] Fixed typo
[robertwb] [BEAM-8575] Test flatten a single PC and test flatten a flattened PC
[lukecwik] Allow metrics update to be tolerant to uninitalized metric containers.
[lukecwik] [BEAM-3493] Prevent users from "implementing" PipelineOptions. (#10005)
[robertwb] [BEAM-8645] Create a py test case for Re-iteration on GBK result.
[suztomo] Correct value is 'Triage Needed'
[suztomo] Reverting unnecessary changes
[robertwb] More compartmentalization of bundle-based-runner only utilities.
[robertwb] [BEAM-8335] Adds the StreamingCache (#10119)
[github] [BEAM-8151] Further cleanup of SDK Workers. (#10134)
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3
Commit message: "Merge pull request #10160 More compartmentalization of bundle-based-runner only utilities."
> git rev-list --no-walk c240f06c4c17435aa5b93f6a599ca17bb4ef7ae6 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6968269511515767424.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4541166570672216435.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3844286911593489403.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5635265880509185082.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8701828862573023002.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2961273150916474028.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/0d/24/35da9c690c1d5042d45c40bbce49daf65e8a754b38f78f8c3d394bd12843/google_cloud_monitoring-0.33.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.33.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3507928308017376558.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-20 01:28:46,582 932af8fb MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/932af8fb/pkb.log>
2019-11-20 01:28:46,583 932af8fb MainThread INFO PerfKitBenchmarker version: v1.12.0-1532-g2784642
2019-11-20 01:28:46,584 932af8fb MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-20 01:28:47,110 932af8fb MainThread INFO Setting --max_concurrent_threads=200.
2019-11-20 01:28:47,134 932af8fb MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-20 01:28:47,189 932af8fb MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-20 01:28:47,192 932af8fb MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-20 01:28:47,192 932af8fb MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-20 01:28:47,193 932af8fb MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 01:28:47,193 932af8fb MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-20 01:28:47,198 932af8fb MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 01:28:47,198 932af8fb MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-20 01:28:47,198 932af8fb MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-20 01:28:47,198 932af8fb MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/932af8fb/pkb.log>
2019-11-20 01:28:47,198 932af8fb MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/932af8fb/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_WordCountIT_Py27 #782
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/782/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#781
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/781/display/redirect?page=changes>
Changes:
[suztomo] Dataflow Java worker to avoid undeclared Guava
[suztomo] Beam SQL JDBC driver not to declare unused Guava
[suztomo] KinesisIO to declare Guava dependency
[suztomo] ZetaSQL to declare Guava dependency
[suztomo] Removed unused dependency from elasticsearch-tests-2
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 1988284a89b10b60eea48325f8a3b370b551c77c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 1988284a89b10b60eea48325f8a3b370b551c77c
Commit message: "Merge pull request #10172: [BEAM-8747] Guava dependency cleanup"
> git rev-list --no-walk 6a921b879384b3f36d8d8447d82c7538699826aa # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5661407924158579036.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4195683941484293473.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins320459172417410397.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2461321367056007251.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7392766988560323633.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3364935154705285356.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Downloading https://files.pythonhosted.org/packages/15/c8/b55cff970507d28cca487b644c738effbd4cbfb4065318f2e6b67f27944f/protobuf-3.11.0-cp27-cp27mu-manylinux1_x86_64.whl (1.3MB)
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.11.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins9038315915476755728.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-26 06:19:46,654 d22a79f4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/d22a79f4/pkb.log>
2019-11-26 06:19:46,654 d22a79f4 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-26 06:19:46,656 d22a79f4 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-26 06:19:47,103 d22a79f4 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-26 06:19:47,128 d22a79f4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-26 06:19:47,186 d22a79f4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-26 06:19:47,189 d22a79f4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-26 06:19:47,189 d22a79f4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-26 06:19:47,190 d22a79f4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 06:19:47,191 d22a79f4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-26 06:19:47,192 d22a79f4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 06:19:47,192 d22a79f4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-26 06:19:47,192 d22a79f4 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-26 06:19:47,192 d22a79f4 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/d22a79f4/pkb.log>
2019-11-26 06:19:47,192 d22a79f4 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/d22a79f4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#780
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/780/display/redirect?page=changes>
Changes:
[worldkzd] fix typos
[github] Update class_test.go
[worldkzd] keep 'a https'
[pabloem] [BEAM-876] Support schemaUpdateOption in BigQueryIO (#9524)
[robertwb] [BEAM-8575] Added a unit test that Reshuffle preserves timestamps
[thw] [BEAM-8815] Skip manifest when no artifacts are staged
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6a921b879384b3f36d8d8447d82c7538699826aa (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6a921b879384b3f36d8d8447d82c7538699826aa
Commit message: "Merge pull request #10205: [BEAM-8815] Skip manifest when no artifacts are staged"
> git rev-list --no-walk e3e52ad55b290cdf9422ccc37f09835c6ee093bc # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins594836289866638630.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins9081449609846179158.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8996500836340338477.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8794146186763817885.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins402120871194058644.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1002555874225236195.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5272403790194086417.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-26 00:39:06,499 56eecef3 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/56eecef3/pkb.log>
2019-11-26 00:39:06,500 56eecef3 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-26 00:39:06,501 56eecef3 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-26 00:39:07,095 56eecef3 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-26 00:39:07,120 56eecef3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-26 00:39:07,183 56eecef3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-26 00:39:07,186 56eecef3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-26 00:39:07,186 56eecef3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-26 00:39:07,187 56eecef3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 00:39:07,188 56eecef3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-26 00:39:07,189 56eecef3 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 00:39:07,189 56eecef3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-26 00:39:07,190 56eecef3 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-26 00:39:07,190 56eecef3 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/56eecef3/pkb.log>
2019-11-26 00:39:07,190 56eecef3 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/56eecef3/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#779
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/779/display/redirect?page=changes>
Changes:
[kirillkozlov] Fix MongoDb SQL Integration Tests
[kirillkozlov] Add MongoDbIT back to build file
[kirillkozlov] Update JavaDoc comment and remove pipeline options
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e3e52ad55b290cdf9422ccc37f09835c6ee093bc (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e3e52ad55b290cdf9422ccc37f09835c6ee093bc
Commit message: "Merge pull request #10061: [BEAM-8586] [SQL] Fix MongoDb integration tests"
> git rev-list --no-walk 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2113848607567381471.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8641641405159746759.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4797691063476029710.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins376455194572881866.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6708844829024817555.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6907247129771808568.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3922427742326271187.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 18:10:39,036 ac4cbe7c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/ac4cbe7c/pkb.log>
2019-11-25 18:10:39,037 ac4cbe7c MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 18:10:39,038 ac4cbe7c MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 18:10:39,622 ac4cbe7c MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 18:10:39,646 ac4cbe7c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 18:10:39,706 ac4cbe7c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 18:10:39,709 ac4cbe7c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 18:10:39,709 ac4cbe7c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 18:10:39,710 ac4cbe7c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 18:10:39,711 ac4cbe7c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 18:10:39,712 ac4cbe7c MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 18:10:39,712 ac4cbe7c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 18:10:39,713 ac4cbe7c MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 18:10:39,713 ac4cbe7c MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/ac4cbe7c/pkb.log>
2019-11-25 18:10:39,713 ac4cbe7c MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/ac4cbe7c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#778
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/778/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 885ecbf3f49257e8b6b4ac376cb5a79ed6282580
Commit message: "[BEAM-8803] BigQuery Streaming Inserts are always retried by default. (#10195)"
> git rev-list --no-walk 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins237975094414831456.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins579188088452442518.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins724576209384970316.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7679878639085614446.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7013285808516981066.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3833131406521531409.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7906269431993523006.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 12:10:27,657 be4c6e29 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/be4c6e29/pkb.log>
2019-11-25 12:10:27,658 be4c6e29 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 12:10:27,659 be4c6e29 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 12:10:28,179 be4c6e29 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 12:10:28,204 be4c6e29 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 12:10:28,264 be4c6e29 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 12:10:28,267 be4c6e29 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 12:10:28,267 be4c6e29 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 12:10:28,268 be4c6e29 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 12:10:28,269 be4c6e29 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 12:10:28,270 be4c6e29 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 12:10:28,270 be4c6e29 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 12:10:28,270 be4c6e29 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 12:10:28,271 be4c6e29 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/be4c6e29/pkb.log>
2019-11-25 12:10:28,271 be4c6e29 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/be4c6e29/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#777
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/777/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 885ecbf3f49257e8b6b4ac376cb5a79ed6282580
Commit message: "[BEAM-8803] BigQuery Streaming Inserts are always retried by default. (#10195)"
> git rev-list --no-walk 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4199390057392391855.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins72241650489385389.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5159747578167097717.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5613413733597743147.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2751604228185959223.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5896481080335967850.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8838309086801651714.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 06:26:57,153 eac27873 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/eac27873/pkb.log>
2019-11-25 06:26:57,153 eac27873 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 06:26:57,155 eac27873 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 06:26:58,466 eac27873 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 06:26:58,727 eac27873 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 06:26:59,179 eac27873 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 06:26:59,192 eac27873 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 06:26:59,193 eac27873 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 06:26:59,198 eac27873 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 06:26:59,201 eac27873 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 06:26:59,204 eac27873 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 06:26:59,205 eac27873 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 06:26:59,206 eac27873 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 06:26:59,207 eac27873 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/eac27873/pkb.log>
2019-11-25 06:26:59,207 eac27873 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/eac27873/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#776
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/776/display/redirect?page=changes>
Changes:
[github] [BEAM-8803] BigQuery Streaming Inserts are always retried by default.
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-9 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 885ecbf3f49257e8b6b4ac376cb5a79ed6282580
Commit message: "[BEAM-8803] BigQuery Streaming Inserts are always retried by default. (#10195)"
> git rev-list --no-walk 639fc24b2a1d6886197750fe6be75e8ad8029205 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5304087721785880830.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3815545928535238536.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8672418632455748817.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins581381880447867595.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2106066730332104443.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1513547798877808086.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins840354613287905364.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 00:21:00,634 b17fb770 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/b17fb770/pkb.log>
2019-11-25 00:21:00,634 b17fb770 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 00:21:00,635 b17fb770 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 00:21:01,062 b17fb770 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 00:21:01,093 b17fb770 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 00:21:01,156 b17fb770 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 00:21:01,159 b17fb770 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 00:21:01,159 b17fb770 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 00:21:01,160 b17fb770 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 00:21:01,161 b17fb770 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 00:21:01,162 b17fb770 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 00:21:01,163 b17fb770 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 00:21:01,163 b17fb770 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 00:21:01,163 b17fb770 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/b17fb770/pkb.log>
2019-11-25 00:21:01,163 b17fb770 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/b17fb770/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#775
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/775/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 639fc24b2a1d6886197750fe6be75e8ad8029205 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 639fc24b2a1d6886197750fe6be75e8ad8029205
Commit message: "Merge pull request #10021: [BEAM-8592] Adjusting ZetaSQL table resolution to standard"
> git rev-list --no-walk 639fc24b2a1d6886197750fe6be75e8ad8029205 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7192247308029237843.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7415186933755061775.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4460671832193544655.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7162614517381952395.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins789634218566421731.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6043764042473511431.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins9020596237335184344.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 18:19:04,690 afb6e685 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/afb6e685/pkb.log>
2019-11-24 18:19:04,691 afb6e685 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 18:19:04,692 afb6e685 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 18:19:04,919 afb6e685 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 18:19:04,944 afb6e685 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 18:19:05,006 afb6e685 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 18:19:05,008 afb6e685 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 18:19:05,009 afb6e685 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 18:19:05,010 afb6e685 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 18:19:05,010 afb6e685 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 18:19:05,011 afb6e685 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 18:19:05,012 afb6e685 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 18:19:05,012 afb6e685 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 18:19:05,012 afb6e685 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/afb6e685/pkb.log>
2019-11-24 18:19:05,012 afb6e685 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/afb6e685/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#774
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/774/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-9 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 639fc24b2a1d6886197750fe6be75e8ad8029205 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 639fc24b2a1d6886197750fe6be75e8ad8029205
Commit message: "Merge pull request #10021: [BEAM-8592] Adjusting ZetaSQL table resolution to standard"
> git rev-list --no-walk 639fc24b2a1d6886197750fe6be75e8ad8029205 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins439485627614032804.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5682398831734316759.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7990036560430121114.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3348303161108057682.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2931424339318861233.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8288521301355837202.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1205209885994055634.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 12:23:53,794 12315651 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/12315651/pkb.log>
2019-11-24 12:23:53,795 12315651 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 12:23:53,797 12315651 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 12:23:54,001 12315651 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 12:23:54,025 12315651 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 12:23:54,090 12315651 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 12:23:54,093 12315651 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 12:23:54,093 12315651 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 12:23:54,094 12315651 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 12:23:54,094 12315651 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 12:23:54,096 12315651 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 12:23:54,096 12315651 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 12:23:54,096 12315651 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 12:23:54,096 12315651 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/12315651/pkb.log>
2019-11-24 12:23:54,096 12315651 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/12315651/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#773
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/773/display/redirect?page=changes>
Changes:
[kenn] Use only standard table resolution for ZetaSQL dialect
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-9 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 639fc24b2a1d6886197750fe6be75e8ad8029205 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 639fc24b2a1d6886197750fe6be75e8ad8029205
Commit message: "Merge pull request #10021: [BEAM-8592] Adjusting ZetaSQL table resolution to standard"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4510081860341142979.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8718721355478172726.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins784459657854937628.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6552302054524355084.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1742683546279169151.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7323809610117158191.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3839389651729684851.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 06:14:44,743 a526908a MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/a526908a/pkb.log>
2019-11-24 06:14:44,743 a526908a MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 06:14:44,744 a526908a MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 06:14:45,125 a526908a MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 06:14:45,149 a526908a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 06:14:45,212 a526908a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 06:14:45,215 a526908a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 06:14:45,215 a526908a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 06:14:45,216 a526908a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 06:14:45,217 a526908a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 06:14:45,218 a526908a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 06:14:45,218 a526908a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 06:14:45,218 a526908a MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 06:14:45,219 a526908a MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/a526908a/pkb.log>
2019-11-24 06:14:45,219 a526908a MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/a526908a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#772
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/772/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7361545881866052043.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1002225869399423937.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1751974923755482277.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins626591929731275279.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3387571369697861759.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7151667357213813657.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3207988825225798300.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 00:19:40,094 135b9baa MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/135b9baa/pkb.log>
2019-11-24 00:19:40,095 135b9baa MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 00:19:40,096 135b9baa MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 00:19:40,577 135b9baa MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 00:19:40,600 135b9baa MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 00:19:40,650 135b9baa MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 00:19:40,652 135b9baa MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 00:19:40,652 135b9baa MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 00:19:40,653 135b9baa MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 00:19:40,654 135b9baa MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 00:19:40,655 135b9baa MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 00:19:40,655 135b9baa MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 00:19:40,655 135b9baa MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 00:19:40,655 135b9baa MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/135b9baa/pkb.log>
2019-11-24 00:19:40,655 135b9baa MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/135b9baa/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#771
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/771/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins9095430039284911125.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1348834158426417701.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2501524182469939695.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins293753660376600232.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1589382878162873664.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1351435640589510285.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2802474943463234088.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 18:19:40,516 bd72d2b5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/bd72d2b5/pkb.log>
2019-11-23 18:19:40,516 bd72d2b5 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 18:19:40,517 bd72d2b5 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 18:19:40,898 bd72d2b5 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 18:19:40,922 bd72d2b5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 18:19:40,972 bd72d2b5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 18:19:40,975 bd72d2b5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 18:19:40,975 bd72d2b5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 18:19:40,976 bd72d2b5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 18:19:40,977 bd72d2b5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 18:19:40,978 bd72d2b5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 18:19:40,978 bd72d2b5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 18:19:40,978 bd72d2b5 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 18:19:40,978 bd72d2b5 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/bd72d2b5/pkb.log>
2019-11-23 18:19:40,979 bd72d2b5 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/bd72d2b5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#770
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/770/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5203785114239940537.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6018016552892773318.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5372291780255654302.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7558021535087681445.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins699009554258431941.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2550256021946560878.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2024694445702830347.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 12:23:42,901 83f7e4ce MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/83f7e4ce/pkb.log>
2019-11-23 12:23:42,902 83f7e4ce MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 12:23:42,903 83f7e4ce MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 12:23:43,482 83f7e4ce MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 12:23:43,506 83f7e4ce MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 12:23:43,558 83f7e4ce MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 12:23:43,560 83f7e4ce MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 12:23:43,560 83f7e4ce MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 12:23:43,561 83f7e4ce MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 12:23:43,562 83f7e4ce MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 12:23:43,563 83f7e4ce MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 12:23:43,563 83f7e4ce MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 12:23:43,563 83f7e4ce MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 12:23:43,564 83f7e4ce MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/83f7e4ce/pkb.log>
2019-11-23 12:23:43,564 83f7e4ce MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/83f7e4ce/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#769
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/769/display/redirect?page=changes>
Changes:
[tweise] Documenting how to use Beam twitter handle (#10082)
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a8f319582713899d5b07aeef0b9e3aed059c0d82 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8578738888653503739.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5441329655627454273.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1135729233844904287.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6950112246044105273.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3261832568351028002.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins9105441582634020945.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2782809997247245808.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 06:21:18,546 8598bcf2 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/8598bcf2/pkb.log>
2019-11-23 06:21:18,547 8598bcf2 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 06:21:18,549 8598bcf2 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 06:21:19,111 8598bcf2 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 06:21:19,136 8598bcf2 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 06:21:19,198 8598bcf2 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 06:21:19,200 8598bcf2 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 06:21:19,201 8598bcf2 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 06:21:19,202 8598bcf2 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 06:21:19,202 8598bcf2 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 06:21:19,203 8598bcf2 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 06:21:19,204 8598bcf2 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 06:21:19,204 8598bcf2 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 06:21:19,204 8598bcf2 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/8598bcf2/pkb.log>
2019-11-23 06:21:19,204 8598bcf2 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/8598bcf2/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#768
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/768/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-8802] Don't clear watermark hold when adding elements.
[kcweaver] [BEAM-8805] Remove obsolete worker_threads experiment in tests
[sunjincheng121] [BEAM-8619] Move reusable information to BundleProcessor.
[sunjincheng121] [BEAM-8619] Extract tearDown functions into BundleProcessor.
[sunjincheng121] [BEAM-8619] Reuse the BundleProcessor between bundles for the same
[sunjincheng121] [BEAM-8619] Teardown the DoFns when upon control service termination for
[lukecwik] [BEAM-7948] Add time-based cache threshold support in the Java data s…
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a8f319582713899d5b07aeef0b9e3aed059c0d82 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a8f319582713899d5b07aeef0b9e3aed059c0d82
Commit message: "[BEAM-8805] Remove obsolete worker_threads experiment in tests"
> git rev-list --no-walk 543abed051a58206c59b44a65e94251a9b9c6d14 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6854425851091126745.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5056909399255448733.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1601926088792996674.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3223285531684140606.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7387187954812965142.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1684330023658346780.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3128637645576687928.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 00:26:27,090 6200262d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/6200262d/pkb.log>
2019-11-23 00:26:27,090 6200262d MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 00:26:27,092 6200262d MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 00:26:27,599 6200262d MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 00:26:27,624 6200262d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 00:26:27,695 6200262d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 00:26:27,697 6200262d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 00:26:27,698 6200262d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 00:26:27,699 6200262d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 00:26:27,699 6200262d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 00:26:27,700 6200262d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 00:26:27,701 6200262d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 00:26:27,701 6200262d MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 00:26:27,701 6200262d MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/6200262d/pkb.log>
2019-11-23 00:26:27,701 6200262d MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/6200262d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#767
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/767/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-8658] [BEAM-8781] Optionally set jar and artifact staging port in
[kcweaver] Pass artifact port to FlinkJarJobServer as well.
[kcweaver] Move FlinkRunnerOptions to pipeline_options.
[kcweaver] [BEAM-8796] Optionally configure static ports for job and expansion
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 543abed051a58206c59b44a65e94251a9b9c6d14 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 543abed051a58206c59b44a65e94251a9b9c6d14
Commit message: "Merge pull request #10163 from ibzib/artifact-port"
> git rev-list --no-walk 07d952f313477ee18cdc706100ba7e1810b1ef4f # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3868107297381231569.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1903239804080733266.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7475688452936180663.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7869946959950981280.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6224270408552318507.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1808810456092303560.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4498681384025679688.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 18:27:35,855 fbb020ed MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/fbb020ed/pkb.log>
2019-11-22 18:27:35,855 fbb020ed MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 18:27:35,857 fbb020ed MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 18:27:36,234 fbb020ed MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 18:27:36,259 fbb020ed MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 18:27:36,323 fbb020ed MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 18:27:36,326 fbb020ed MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 18:27:36,326 fbb020ed MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 18:27:36,327 fbb020ed MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 18:27:36,328 fbb020ed MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 18:27:36,329 fbb020ed MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 18:27:36,330 fbb020ed MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 18:27:36,330 fbb020ed MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 18:27:36,330 fbb020ed MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/fbb020ed/pkb.log>
2019-11-22 18:27:36,330 fbb020ed MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/fbb020ed/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#766
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/766/display/redirect?page=changes>
Changes:
[33895511+aromanenko-dev] [BEAM-7636] Migrate SqsIO to AWS SDK V2 for Java
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 07d952f313477ee18cdc706100ba7e1810b1ef4f (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 07d952f313477ee18cdc706100ba7e1810b1ef4f
Commit message: "[BEAM-7636] Migrate SqsIO to AWS SDK V2 for Java"
> git rev-list --no-walk 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3608175143305367111.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7306957321880756505.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3693865905449557898.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins366387556087109335.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2768785821828006802.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1734185207142085832.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6547157599378435175.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 12:23:24,646 617d2deb MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/617d2deb/pkb.log>
2019-11-22 12:23:24,646 617d2deb MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 12:23:24,647 617d2deb MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 12:23:25,034 617d2deb MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 12:23:25,058 617d2deb MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 12:23:25,111 617d2deb MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 12:23:25,113 617d2deb MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 12:23:25,114 617d2deb MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 12:23:25,115 617d2deb MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 12:23:25,115 617d2deb MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 12:23:25,116 617d2deb MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 12:23:25,117 617d2deb MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 12:23:25,117 617d2deb MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 12:23:25,117 617d2deb MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/617d2deb/pkb.log>
2019-11-22 12:23:25,117 617d2deb MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/617d2deb/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#765
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/765/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb
Commit message: "[BEAM-7278, BEAM-2530] Add support for using a Java linkage testing tool to aid upgrading dependencies. (#10184)"
> git rev-list --no-walk 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2954991931250393667.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5120730772775615016.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3141298636529857083.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3111850956201268291.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5668493566784248394.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins315506324720854569.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7151272358559824541.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 06:23:12,308 65f23b1d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/65f23b1d/pkb.log>
2019-11-22 06:23:12,309 65f23b1d MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 06:23:12,311 65f23b1d MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 06:23:12,668 65f23b1d MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 06:23:12,695 65f23b1d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 06:23:12,791 65f23b1d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 06:23:12,795 65f23b1d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 06:23:12,795 65f23b1d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 06:23:12,796 65f23b1d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 06:23:12,797 65f23b1d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 06:23:12,798 65f23b1d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 06:23:12,799 65f23b1d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 06:23:12,799 65f23b1d MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 06:23:12,799 65f23b1d MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/65f23b1d/pkb.log>
2019-11-22 06:23:12,799 65f23b1d MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/65f23b1d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#764
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/764/display/redirect?page=changes>
Changes:
[github] [BEAM-8659] RowJsonTest should test serialization independently
[github] [BEAM-8743] Add support for flat schemas in pubsub
[pabloem] [BEAM-8016] Pipeline Graph (#10132)
[github] [BEAM-7278, BEAM-2530] Add support for using a Java linkage testing tool
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb
Commit message: "[BEAM-7278, BEAM-2530] Add support for using a Java linkage testing tool to aid upgrading dependencies. (#10184)"
> git rev-list --no-walk 07a5155974a72b336d686efc77cc8a6dd71aa581 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6908603120421486523.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins664898933531805833.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8468532215071856369.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6484735691797875337.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins657897272423931915.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7929262858099549807.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1091956279306577230.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 00:46:12,367 c77daaff MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/c77daaff/pkb.log>
2019-11-22 00:46:12,368 c77daaff MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 00:46:12,369 c77daaff MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 00:46:13,022 c77daaff MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 00:46:13,094 c77daaff MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 00:46:13,183 c77daaff MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 00:46:13,187 c77daaff MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 00:46:13,188 c77daaff MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 00:46:13,190 c77daaff MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 00:46:13,191 c77daaff MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 00:46:13,193 c77daaff MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 00:46:13,193 c77daaff MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 00:46:13,194 c77daaff MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 00:46:13,194 c77daaff MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/c77daaff/pkb.log>
2019-11-22 00:46:13,194 c77daaff MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/c77daaff/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#763
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/763/display/redirect?page=changes>
Changes:
[bhulette] Fix inverted timeout check in TestPubsub
[bhulette] Add empty check before acknowledge
[echauchot] [BEAM-8470] Enable custom window tests in new spark runner validates
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-13 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 07a5155974a72b336d686efc77cc8a6dd71aa581 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 07a5155974a72b336d686efc77cc8a6dd71aa581
Commit message: "Merge pull request #10153: [BEAM-8740] TestPubsub ignores timeout"
> git rev-list --no-walk cd36c75174ed0f1be806b8aa7a4819bacbededb7 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4142874828350698203.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2627554360070525576.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4728924178040825237.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1546562351553287918.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2597937908393568144.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8898412029895762522.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7207521120217286931.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 18:33:35,339 06cd3222 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/06cd3222/pkb.log>
2019-11-21 18:33:35,340 06cd3222 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 18:33:35,341 06cd3222 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 18:33:35,807 06cd3222 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 18:33:35,831 06cd3222 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 18:33:35,895 06cd3222 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 18:33:35,897 06cd3222 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 18:33:35,898 06cd3222 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 18:33:35,899 06cd3222 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 18:33:35,899 06cd3222 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 18:33:35,900 06cd3222 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 18:33:35,901 06cd3222 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 18:33:35,901 06cd3222 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 18:33:35,901 06cd3222 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/06cd3222/pkb.log>
2019-11-21 18:33:35,901 06cd3222 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/06cd3222/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#762
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/762/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision cd36c75174ed0f1be806b8aa7a4819bacbededb7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f cd36c75174ed0f1be806b8aa7a4819bacbededb7
Commit message: "Merge pull request #10181: Website: "common" --> "unique" (and other word choice fixes)"
> git rev-list --no-walk cd36c75174ed0f1be806b8aa7a4819bacbededb7 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4049156177232013149.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6384729616782892378.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7647706299540775917.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5699030098456898302.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3107348751149556620.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins457893842520457123.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3959940819088764395.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 12:25:27,689 fda01793 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/fda01793/pkb.log>
2019-11-21 12:25:27,689 fda01793 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 12:25:27,690 fda01793 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 12:25:28,142 fda01793 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 12:25:28,167 fda01793 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 12:25:28,220 fda01793 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 12:25:28,223 fda01793 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 12:25:28,223 fda01793 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 12:25:28,224 fda01793 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 12:25:28,225 fda01793 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 12:25:28,226 fda01793 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 12:25:28,226 fda01793 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 12:25:28,226 fda01793 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 12:25:28,227 fda01793 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/fda01793/pkb.log>
2019-11-21 12:25:28,227 fda01793 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/fda01793/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#761
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/761/display/redirect?page=changes>
Changes:
[github] common --> unique
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision cd36c75174ed0f1be806b8aa7a4819bacbededb7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f cd36c75174ed0f1be806b8aa7a4819bacbededb7
Commit message: "Merge pull request #10181: Website: "common" --> "unique" (and other word choice fixes)"
> git rev-list --no-walk 5e3adf5e25af372253da42a28e8318b6f535c063 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3955473405321862962.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5625563155605071311.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1804436340632379311.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1768908515958699162.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4449432118128677874.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8091455805370545481.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7923358547661779879.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 06:25:46,396 a7648581 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/a7648581/pkb.log>
2019-11-21 06:25:46,396 a7648581 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 06:25:46,398 a7648581 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 06:25:46,775 a7648581 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 06:25:46,802 a7648581 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 06:25:46,869 a7648581 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 06:25:46,871 a7648581 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 06:25:46,872 a7648581 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 06:25:46,873 a7648581 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 06:25:46,874 a7648581 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 06:25:46,875 a7648581 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 06:25:46,876 a7648581 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 06:25:46,876 a7648581 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 06:25:46,876 a7648581 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/a7648581/pkb.log>
2019-11-21 06:25:46,876 a7648581 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/a7648581/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#760
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/760/display/redirect?page=changes>
Changes:
[M.Yzvenn] fix - 1MB is interpreted as 1000, not 1024
[robertwb] [BEAM-8629] Don't return mutable class type hints.
[valentyn] Guard pickling operations with a lock to prevent race condition in
[tvalentyn] [BEAM-8575] Add a Python test to test windowing in DoFn finish_bundle()
[github] [BEAM-3419] Flesh out iterable side inputs and key enumeration for
[kcweaver] [BEAM-8795] fix Spark runner build
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5e3adf5e25af372253da42a28e8318b6f535c063 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 5e3adf5e25af372253da42a28e8318b6f535c063
Commit message: "Merge pull request #10167 [BEAM-8651] Guard pickling operations with a lock to prevent race condition in module imports."
> git rev-list --no-walk 17e1e8b6682edb3005cf36f2f6a9758fb632cccc # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins600508383119132716.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4088226321175785201.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1578743202233846820.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4159395557892897175.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1754710165514761214.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6303254073367226206.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/0d/24/35da9c690c1d5042d45c40bbce49daf65e8a754b38f78f8c3d394bd12843/google_cloud_monitoring-0.33.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.33.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3347140552043954979.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 00:45:22,173 54aede03 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/54aede03/pkb.log>
2019-11-21 00:45:22,174 54aede03 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 00:45:22,175 54aede03 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 00:45:22,624 54aede03 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 00:45:22,649 54aede03 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 00:45:22,705 54aede03 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 00:45:22,708 54aede03 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 00:45:22,708 54aede03 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 00:45:22,709 54aede03 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 00:45:22,710 54aede03 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 00:45:22,711 54aede03 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 00:45:22,711 54aede03 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 00:45:22,711 54aede03 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 00:45:22,711 54aede03 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/54aede03/pkb.log>
2019-11-21 00:45:22,711 54aede03 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/54aede03/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#759
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/759/display/redirect?page=changes>
Changes:
[katarzyna.kucharczyk] [BEAM-6335] Added streaming GroupByKey Test that reads SyntheticSource
[katarzyna.kucharczyk] [BEAM-6335] Changed SyntheticDataPublisher to publish String UTF values
[katarzyna.kucharczyk] [BEAM-6335] Added custom PubSub Matcher that stops pipeline after
[valentyn] Restore original behavior of evaluating worker host on Windows until a
[echauchot] [BEAM-8470] Add an empty spark-structured-streaming runner project
[echauchot] [BEAM-8470] Fix missing dep
[echauchot] [BEAM-8470] Add SparkPipelineOptions
[echauchot] [BEAM-8470] Start pipeline translation
[echauchot] [BEAM-8470] Add global pipeline translation structure
[echauchot] [BEAM-8470] Add nodes translators structure
[echauchot] [BEAM-8470] Wire node translators with pipeline translator
[echauchot] [BEAM-8470] Renames: better differenciate pipeline translator for
[echauchot] [BEAM-8470] Organise methods in PipelineTranslator
[echauchot] [BEAM-8470] Initialise BatchTranslationContext
[echauchot] [BEAM-8470] Refactoring: -move batch/streaming common translation
[echauchot] [BEAM-8470] Make transform translation clearer: renaming, comments
[echauchot] [BEAM-8470] Improve javadocs
[echauchot] [BEAM-8470] Move SparkTransformOverrides to correct package
[echauchot] [BEAM-8470] Move common translation context components to superclass
[echauchot] [BEAM-8470] apply spotless
[echauchot] [BEAM-8470] Make codestyle and firebug happy
[echauchot] [BEAM-8470] Add TODOs
[echauchot] [BEAM-8470] Post-pone batch qualifier in all classes names for
[echauchot] [BEAM-8470] Add precise TODO for multiple TransformTranslator per
[echauchot] [BEAM-8470] Added SparkRunnerRegistrar
[echauchot] [BEAM-8470] Add basic pipeline execution. Refactor translatePipeline()
[echauchot] [BEAM-8470] Create PCollections manipulation methods
[echauchot] [BEAM-8470] Create Datasets manipulation methods
[echauchot] [BEAM-8470] Add Flatten transformation translator
[echauchot] [BEAM-8470] Add primitive GroupByKeyTranslatorBatch implementation
[echauchot] [BEAM-8470] Use Iterators.transform() to return Iterable
[echauchot] [BEAM-8470] Implement read transform
[echauchot] [BEAM-8470] update TODO
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] start source instanciation
[echauchot] [BEAM-8470] Improve exception flow
[echauchot] [BEAM-8470] Improve type enforcement in ReadSourceTranslator
[echauchot] [BEAM-8470] Experiment over using spark Catalog to pass in Beam Source
[echauchot] [BEAM-8470] Add source mocks
[echauchot] [BEAM-8470] fix mock, wire mock in translators and create a main test.
[echauchot] [BEAM-8470] Use raw WindowedValue so that spark Encoders could work
[echauchot] [BEAM-8470] clean deps
[echauchot] [BEAM-8470] Move DatasetSourceMock to proper batch mode
[echauchot] [BEAM-8470] Run pipeline in batch mode or in streaming mode
[echauchot] [BEAM-8470] Split batch and streaming sources and translators
[echauchot] [BEAM-8470] Use raw Encoder<WindowedValue> also in regular
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Add ReadSourceTranslatorStreaming
[echauchot] [BEAM-8470] Move Source and translator mocks to a mock package.
[echauchot] [BEAM-8470] Pass Beam Source and PipelineOptions to the spark DataSource
[echauchot] [BEAM-8470] Refactor DatasetSource fields
[echauchot] [BEAM-8470] Wire real SourceTransform and not mock and update the test
[echauchot] [BEAM-8470] Add missing 0-arg public constructor
[echauchot] [BEAM-8470] Use new PipelineOptionsSerializationUtils
[echauchot] [BEAM-8470] Apply spotless and fix checkstyle
[echauchot] [BEAM-8470] Add a dummy schema for reader
[echauchot] [BEAM-8470] Add empty 0-arg constructor for mock source
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Checkstyle and Findbugs
[echauchot] [BEAM-8470] Refactor SourceTest to a UTest instaed of a main
[echauchot] [BEAM-8470] Fix pipeline triggering: use a spark action instead of
[echauchot] [BEAM-8470] improve readability of options passing to the source
[echauchot] [BEAM-8470] Clean unneeded fields in DatasetReader
[echauchot] [BEAM-8470] Fix serialization issues
[echauchot] [BEAM-8470] Add SerializationDebugger
[echauchot] [BEAM-8470] Add serialization test
[echauchot] [BEAM-8470] Move SourceTest to same package as tested class
[echauchot] [BEAM-8470] Fix SourceTest
[echauchot] [BEAM-8470] Simplify beam reader creation as it created once the source
[echauchot] [BEAM-8470] Put all transform translators Serializable
[echauchot] [BEAM-8470] Enable test mode
[echauchot] [BEAM-8470] Enable gradle build scan
[echauchot] [BEAM-8470] Add flatten test
[echauchot] [BEAM-8470] First attempt for ParDo primitive implementation
[echauchot] [BEAM-8470] Serialize windowedValue to byte[] in source to be able to
[echauchot] [BEAM-8470] Comment schema choices
[echauchot] [BEAM-8470] Fix errorprone
[echauchot] [BEAM-8470] Fix testMode output to comply with new binary schema
[echauchot] [BEAM-8470] Cleaning
[echauchot] [BEAM-8470] Remove bundleSize parameter and always use spark default
[echauchot] [BEAM-8470] Fix split bug
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Add ParDoTest
[echauchot] [BEAM-8470] Address minor review notes
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Add GroupByKeyTest
[echauchot] [BEAM-8470] Add comments and TODO to GroupByKeyTranslatorBatch
[echauchot] [BEAM-8470] Fix type checking with Encoder of WindowedValue<T>
[echauchot] [BEAM-8470] Port latest changes of ReadSourceTranslatorBatch to
[echauchot] [BEAM-8470] Remove no more needed putDatasetRaw
[echauchot] [BEAM-8470] Add ComplexSourceTest
[echauchot] [BEAM-8470] Fail in case of having SideInouts or State/Timers
[echauchot] [BEAM-8470] Fix Encoders: create an Encoder for every manipulated type
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] Fixed Javadoc error
[echauchot] [BEAM-8470] Rename SparkSideInputReader class and rename pruneOutput()
[echauchot] [BEAM-8470] Don't use deprecated
[echauchot] [BEAM-8470] Simplify logic of ParDo translator
[echauchot] [BEAM-8470] Fix kryo issue in GBK translator with a workaround
[echauchot] [BEAM-8470] Rename SparkOutputManager for consistency
[echauchot] [BEAM-8470] Fix for test elements container in GroupByKeyTest
[echauchot] [BEAM-8470] Added "testTwoPardoInRow"
[echauchot] [BEAM-8470] Add a test for the most simple possible Combine
[echauchot] [BEAM-8470] Rename SparkDoFnFilterFunction to DoFnFilterFunction for
[echauchot] [BEAM-8470] Generalize the use of SerializablePipelineOptions in place
[echauchot] [BEAM-8470] Fix getSideInputs
[echauchot] [BEAM-8470] Extract binary schema creation in a helper class
[echauchot] [BEAM-8470] First version of combinePerKey
[echauchot] [BEAM-8470] Improve type checking of Tuple2 encoder
[echauchot] [BEAM-8470] Introduce WindowingHelpers (and helpers package) and use it
[echauchot] [BEAM-8470] Fix combiner using KV as input, use binary encoders in place
[echauchot] [BEAM-8470] Add combinePerKey and CombineGlobally tests
[echauchot] [BEAM-8470] Introduce RowHelpers
[echauchot] [BEAM-8470] Add CombineGlobally translation to avoid translating
[echauchot] [BEAM-8470] Cleaning
[echauchot] [BEAM-8470] Get back to classes in translators resolution because URNs
[echauchot] [BEAM-8470] Fix various type checking issues in Combine.Globally
[echauchot] [BEAM-8470] Update test with Long
[echauchot] [BEAM-8470] Fix combine. For unknown reason GenericRowWithSchema is used
[echauchot] [BEAM-8470] Use more generic Row instead of GenericRowWithSchema
[echauchot] [BEAM-8470] Add explanation about receiving a Row as input in the
[echauchot] [BEAM-8470] Fix encoder bug in combinePerkey
[echauchot] [BEAM-8470] Cleaning
[echauchot] [BEAM-8470] Implement WindowAssignTranslatorBatch
[echauchot] [BEAM-8470] Implement WindowAssignTest
[echauchot] [BEAM-8470] Fix javadoc
[echauchot] [BEAM-8470] Added SideInput support
[echauchot] [BEAM-8470] Fix CheckStyle violations
[echauchot] [BEAM-8470] Don't use Reshuffle translation
[echauchot] [BEAM-8470] Added using CachedSideInputReader
[echauchot] [BEAM-8470] Added TODO comment for ReshuffleTranslatorBatch
[echauchot] [BEAM-8470] And unchecked warning suppression
[echauchot] [BEAM-8470] Add streaming source initialisation
[echauchot] [BEAM-8470] Implement first streaming source
[echauchot] [BEAM-8470] Add a TODO on spark output modes
[echauchot] [BEAM-8470] Add transformators registry in PipelineTranslatorStreaming
[echauchot] [BEAM-8470] Add source streaming test
[echauchot] [BEAM-8470] Specify checkpointLocation at the pipeline start
[echauchot] [BEAM-8470] Clean unneeded 0 arg constructor in batch source
[echauchot] [BEAM-8470] Clean streaming source
[echauchot] [BEAM-8470] Continue impl of offsets for streaming source
[echauchot] [BEAM-8470] Deal with checkpoint and offset based read
[echauchot] [BEAM-8470] Apply spotless and fix spotbugs warnings
[echauchot] [BEAM-8470] Disable never ending test
[echauchot] [BEAM-8470] Fix access level issues, typos and modernize code to Java 8
[echauchot] [BEAM-8470] Merge Spark Structured Streaming runner into main Spark
[echauchot] [BEAM-8470] Fix non-vendored imports from Spark Streaming Runner classes
[echauchot] [BEAM-8470] Pass doFnSchemaInformation to ParDo batch translation
[echauchot] [BEAM-8470] Fix spotless issues after rebase
[echauchot] [BEAM-8470] Fix logging levels in Spark Structured Streaming translation
[echauchot] [BEAM-8470] Add SparkStructuredStreamingPipelineOptions and
[echauchot] [BEAM-8470] Rename SparkPipelineResult to
[echauchot] [BEAM-8470] Use PAssert in Spark Structured Streaming transform tests
[echauchot] [BEAM-8470] Ignore spark offsets (cf javadoc)
[echauchot] [BEAM-8470] implement source.stop
[echauchot] [BEAM-8470] Update javadoc
[echauchot] [BEAM-8470] Apply Spotless
[echauchot] [BEAM-8470] Enable batch Validates Runner tests for Structured Streaming
[echauchot] [BEAM-8470] Limit the number of partitions to make tests go 300% faster
[echauchot] [BEAM-8470] Fixes ParDo not calling setup and not tearing down if
[echauchot] [BEAM-8470] Pass transform based doFnSchemaInformation in ParDo
[echauchot] [BEAM-8470] Consider null object case on RowHelpers, fixes empty side
[echauchot] [BEAM-8470] Put back batch/simpleSourceTest.testBoundedSource
[echauchot] [BEAM-8470] Update windowAssignTest
[echauchot] [BEAM-8470] Add comment about checkpoint mark
[echauchot] [BEAM-8470] Re-code GroupByKeyTranslatorBatch to conserve windowing
[echauchot] [BEAM-8470] re-enable reduceFnRunner timers for output
[echauchot] [BEAM-8470] Improve visibility of debug messages
[echauchot] [BEAM-8470] Add a test that GBK preserves windowing
[echauchot] [BEAM-8470] Add TODO in Combine translations
[echauchot] [BEAM-8470] Update KVHelpers.extractKey() to deal with WindowedValue and
[echauchot] [BEAM-8470] Fix comment about schemas
[echauchot] [BEAM-8470] Implement reduce part of CombineGlobally translation with
[echauchot] [BEAM-8470] Output data after combine
[echauchot] [BEAM-8470] Implement merge accumulators part of CombineGlobally
[echauchot] [BEAM-8470] Fix encoder in combine call
[echauchot] [BEAM-8470] Revert extractKey while combinePerKey is not done (so that
[echauchot] [BEAM-8470] Apply a groupByKey avoids for some reason that the spark
[echauchot] [BEAM-8470] Fix case when a window does not merge into any other window
[echauchot] [BEAM-8470] Fix wrong encoder in combineGlobally GBK
[echauchot] [BEAM-8470] Fix bug in the window merging logic
[echauchot] [BEAM-8470] Remove the mapPartition that adds a key per partition
[echauchot] [BEAM-8470] Remove CombineGlobally translation because it is less
[echauchot] [BEAM-8470] Now that there is only Combine.PerKey translation, make only
[echauchot] [BEAM-8470] Clean no more needed KVHelpers
[echauchot] [BEAM-8470] Clean not more needed RowHelpers
[echauchot] [BEAM-8470] Clean not more needed WindowingHelpers
[echauchot] [BEAM-8470] Fix javadoc of AggregatorCombiner
[echauchot] [BEAM-8470] Fixed immutable list bug
[echauchot] [BEAM-8470] add comment in combine globally test
[echauchot] [BEAM-8470] Clean groupByKeyTest
[echauchot] [BEAM-8470] Add a test that combine per key preserves windowing
[echauchot] [BEAM-8470] Ignore for now not working test testCombineGlobally
[echauchot] [BEAM-8470] Add metrics support in DoFn
[echauchot] [BEAM-8470] Add missing dependencies to run Spark Structured Streaming
[echauchot] [BEAM-8470] Add setEnableSparkMetricSinks() method
[echauchot] [BEAM-8470] Fix javadoc
[echauchot] [BEAM-8470] Fix accumulators initialization in Combine that prevented
[echauchot] [BEAM-8470] Add a test to check that CombineGlobally preserves windowing
[echauchot] [BEAM-8470] Persist all output Dataset if there are multiple outputs in
[echauchot] [BEAM-8470] Added metrics sinks and tests
[echauchot] [BEAM-8470] Make spotless happy
[echauchot] [BEAM-8470] Add PipelineResults to Spark structured streaming.
[echauchot] [BEAM-8470] Update log4j configuration
[echauchot] [BEAM-8470] Add spark execution plans extended debug messages.
[echauchot] [BEAM-8470] Print number of leaf datasets
[echauchot] [BEAM-8470] fixup! Add PipelineResults to Spark structured streaming.
[echauchot] [BEAM-8470] Remove no more needed AggregatorCombinerPerKey (there is
[echauchot] [BEAM-8470] After testing performance and correctness, launch pipeline
[echauchot] [BEAM-8470] Improve Pardo translation performance: avoid calling a
[echauchot] [BEAM-8470] Use "sparkMaster" in local mode to obtain number of shuffle
[echauchot] [BEAM-8470] Wrap Beam Coders into Spark Encoders using
[echauchot] [BEAM-8470] Wrap Beam Coders into Spark Encoders using
[echauchot] [BEAM-8470] type erasure: spark encoders require a Class<T>, pass Object
[echauchot] [BEAM-8470] Fix scala Product in Encoders to avoid StackEverflow
[echauchot] [BEAM-8470] Conform to spark ExpressionEncoders: pass classTags,
[echauchot] [BEAM-8470] Add a simple spark native test to test Beam coders wrapping
[echauchot] [BEAM-8470] Fix code generation in Beam coder wrapper
[echauchot] [BEAM-8470] Lazy init coder because coder instance cannot be
[echauchot] [BEAM-8470] Fix warning in coder construction by reflexion
[echauchot] [BEAM-8470] Fix ExpressionEncoder generated code: typos, try catch, fqcn
[echauchot] [BEAM-8470] Fix getting the output value in code generation
[echauchot] [BEAM-8470] Fix beam coder lazy init using reflexion: use .clas + try
[echauchot] [BEAM-8470] Remove lazy init of beam coder because there is no generic
[echauchot] [BEAM-8470] Remove example code
[echauchot] [BEAM-8470] Fix equal and hashcode
[echauchot] [BEAM-8470] Fix generated code: uniform exceptions catching, fix
[echauchot] [BEAM-8470] Add an assert of equality in the encoders test
[echauchot] [BEAM-8470] Apply spotless and checkstyle and add javadocs
[echauchot] [BEAM-8470] Wrap exceptions in UserCoderExceptions
[echauchot] [BEAM-8470] Put Encoders expressions serializable
[echauchot] [BEAM-8470] Catch Exception instead of IOException because some coders
[echauchot] [BEAM-8470] Apply new Encoders to CombinePerKey
[echauchot] [BEAM-8470] Apply new Encoders to Read source
[echauchot] [BEAM-8470] Improve performance of source: the mapper already calls
[echauchot] [BEAM-8470] Ignore long time failing test: SparkMetricsSinkTest
[echauchot] [BEAM-8470] Apply new Encoders to Window assign translation
[echauchot] [BEAM-8470] Apply new Encoders to AggregatorCombiner
[echauchot] [BEAM-8470] Create a Tuple2Coder to encode scala tuple2
[echauchot] [BEAM-8470] Apply new Encoders to GroupByKey
[echauchot] [BEAM-8470] Apply new Encoders to Pardo. Replace Tuple2Coder with
[echauchot] [BEAM-8470] Apply spotless, fix typo and javadoc
[echauchot] [BEAM-8470] Use beam encoders also in the output of the source
[echauchot] [BEAM-8470] Remove unneeded cast
[echauchot] [BEAM-8470] Fix: Remove generic hack of using object. Use actual Coder
[echauchot] [BEAM-8470] Remove Encoders based on kryo now that we call Beam coders
[echauchot] [BEAM-8470] Add a jenkins job for validates runner tests in the new
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] Rebase on master: pass sideInputMapping in SimpleDoFnRunner
[echauchot] Fix SpotBugs
[echauchot] [BEAM-8470] simplify coders in combinePerKey translation
[echauchot] [BEAM-8470] Fix combiner. Do not reuse instance of accumulator
[echauchot] [BEAM-8470] input windows can arrive exploded (for sliding windows). As
[echauchot] [BEAM-8470] Add a combine test with sliding windows
[echauchot] [BEAM-8470] Add a test to test combine translation on binaryCombineFn
[echauchot] [BEAM-8470] Fix tests: use correct
[echauchot] [BEAM-8470] Fix wrong expected results in
[echauchot] [BEAM-8470] Add disclaimers about this runner being experimental
[echauchot] [BEAM-8470] Fix: create an empty accumulator in
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] Add a countPerElement test with sliding windows
[echauchot] [BEAM-8470] Fix the output timestamps of combine: timestamps must be
[echauchot] [BEAM-8470] set log level to info to avoid resource consumption in
[echauchot] [BEAM-8470] Fix CombineTest.testCountPerElementWithSlidingWindows
[aromanenko.dev] [BEAM-8470] Remove "validatesStructuredStreamingRunnerBatch" from
[echauchot] [BEAM-8470] Fix timestamps in combine output: assign the timestamp to
[iemejia] [website] Add Spark Structured Runner VR badge to the github template
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 17e1e8b6682edb3005cf36f2f6a9758fb632cccc (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 17e1e8b6682edb3005cf36f2f6a9758fb632cccc
Commit message: "Merge pull request #10110 [BEAM-8646] Restore original behavior of evaluating worker host on Windows until a better solution is available."
> git rev-list --no-walk 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2299101987302454799.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8375229236221560267.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6823509396319797795.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins7719929195473376310.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4465702222319865555.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3210025103006825212.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/0d/24/35da9c690c1d5042d45c40bbce49daf65e8a754b38f78f8c3d394bd12843/google_cloud_monitoring-0.33.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.33.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5838286146798203981.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-20 18:43:49,645 71eff525 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/71eff525/pkb.log>
2019-11-20 18:43:49,645 71eff525 MainThread INFO PerfKitBenchmarker version: v1.12.0-1532-g2784642
2019-11-20 18:43:49,646 71eff525 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-20 18:43:50,088 71eff525 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-20 18:43:50,113 71eff525 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-20 18:43:50,172 71eff525 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-20 18:43:50,174 71eff525 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-20 18:43:50,175 71eff525 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-20 18:43:50,176 71eff525 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 18:43:50,176 71eff525 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-20 18:43:50,177 71eff525 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 18:43:50,178 71eff525 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-20 18:43:50,178 71eff525 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-20 18:43:50,178 71eff525 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/71eff525/pkb.log>
2019-11-20 18:43:50,178 71eff525 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/71eff525/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#758
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/758/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3
Commit message: "Merge pull request #10160 More compartmentalization of bundle-based-runner only utilities."
> git rev-list --no-walk 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins2584271465439046198.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6308043643247797205.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins4083115174527051155.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6844386877294219973.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5135586725550714236.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8960671583943076804.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/0d/24/35da9c690c1d5042d45c40bbce49daf65e8a754b38f78f8c3d394bd12843/google_cloud_monitoring-0.33.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.33.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins1639085103122549374.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-20 12:28:07,625 e6d0c5d8 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/e6d0c5d8/pkb.log>
2019-11-20 12:28:07,626 e6d0c5d8 MainThread INFO PerfKitBenchmarker version: v1.12.0-1532-g2784642
2019-11-20 12:28:07,627 e6d0c5d8 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-20 12:28:08,211 e6d0c5d8 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-20 12:28:08,236 e6d0c5d8 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-20 12:28:08,298 e6d0c5d8 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-20 12:28:08,301 e6d0c5d8 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-20 12:28:08,301 e6d0c5d8 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-20 12:28:08,302 e6d0c5d8 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 12:28:08,303 e6d0c5d8 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-20 12:28:08,304 e6d0c5d8 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 12:28:08,304 e6d0c5d8 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-20 12:28:08,304 e6d0c5d8 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-20 12:28:08,304 e6d0c5d8 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/e6d0c5d8/pkb.log>
2019-11-20 12:28:08,305 e6d0c5d8 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/e6d0c5d8/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py27
#757
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/757/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3
Commit message: "Merge pull request #10160 More compartmentalization of bundle-based-runner only utilities."
> git rev-list --no-walk 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3244225463569789831.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins928991684673822209.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins5161593232755859927.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins8494177238632343780.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins3991751181161088254.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins6556803930759005132.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/0d/24/35da9c690c1d5042d45c40bbce49daf65e8a754b38f78f8c3d394bd12843/google_cloud_monitoring-0.33.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.33.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py27] $ /bin/bash -xe /tmp/jenkins643411835752449791.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py27_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py2 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-20 06:41:54,366 f31201b3 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/f31201b3/pkb.log>
2019-11-20 06:41:54,367 f31201b3 MainThread INFO PerfKitBenchmarker version: v1.12.0-1532-g2784642
2019-11-20 06:41:54,368 f31201b3 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py2
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py27_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-20 06:41:54,811 f31201b3 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-20 06:41:54,834 f31201b3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-20 06:41:54,886 f31201b3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-20 06:41:54,889 f31201b3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-20 06:41:54,889 f31201b3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-20 06:41:54,890 f31201b3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 06:41:54,890 f31201b3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-20 06:41:54,891 f31201b3 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 06:41:54,892 f31201b3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-20 06:41:54,892 f31201b3 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-20 06:41:54,892 f31201b3 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/f31201b3/pkb.log>
2019-11-20 06:41:54,892 f31201b3 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py27/ws/runs/f31201b3/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org