You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/11/20 12:52:33 UTC
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#729
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/729/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3
Commit message: "Merge pull request #10160 More compartmentalization of bundle-based-runner only utilities."
> git rev-list --no-walk 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8350896810499887614.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7684192429434785262.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins9006685397912831934.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8506298054562267490.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7077967371539152694.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins691467270361235817.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/0d/24/35da9c690c1d5042d45c40bbce49daf65e8a754b38f78f8c3d394bd12843/google_cloud_monitoring-0.33.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.33.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins9104145598927890200.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-20 12:52:32,369 764908c4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/764908c4/pkb.log>
2019-11-20 12:52:32,369 764908c4 MainThread INFO PerfKitBenchmarker version: v1.12.0-1532-g2784642
2019-11-20 12:52:32,371 764908c4 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-20 12:52:32,699 764908c4 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-20 12:52:32,724 764908c4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-20 12:52:32,796 764908c4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-20 12:52:32,798 764908c4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-20 12:52:32,799 764908c4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-20 12:52:32,800 764908c4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 12:52:32,800 764908c4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-20 12:52:32,802 764908c4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 12:52:32,802 764908c4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-20 12:52:32,802 764908c4 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-20 12:52:32,802 764908c4 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/764908c4/pkb.log>
2019-11-20 12:52:32,802 764908c4 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/764908c4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_WordCountIT_Py37 #754
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/754/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#753
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/753/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 1988284a89b10b60eea48325f8a3b370b551c77c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 1988284a89b10b60eea48325f8a3b370b551c77c
Commit message: "Merge pull request #10172: [BEAM-8747] Guava dependency cleanup"
> git rev-list --no-walk 1988284a89b10b60eea48325f8a3b370b551c77c # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8463420628478623271.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3797486899456623987.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8615699989844948235.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5845602545566327228.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6311579581029648411.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3975557004529911888.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/15/c8/b55cff970507d28cca487b644c738effbd4cbfb4065318f2e6b67f27944f/protobuf-3.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.11.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2458645627621950667.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-26 12:45:45,149 a835ad2c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/a835ad2c/pkb.log>
2019-11-26 12:45:45,150 a835ad2c MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-26 12:45:45,151 a835ad2c MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-26 12:45:45,776 a835ad2c MainThread INFO Setting --max_concurrent_threads=200.
2019-11-26 12:45:45,802 a835ad2c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-26 12:45:45,867 a835ad2c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-26 12:45:45,870 a835ad2c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-26 12:45:45,870 a835ad2c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-26 12:45:45,871 a835ad2c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 12:45:45,872 a835ad2c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-26 12:45:45,877 a835ad2c MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 12:45:45,877 a835ad2c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-26 12:45:45,878 a835ad2c MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-26 12:45:45,878 a835ad2c MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/a835ad2c/pkb.log>
2019-11-26 12:45:45,878 a835ad2c MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/a835ad2c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#752
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/752/display/redirect?page=changes>
Changes:
[suztomo] Dataflow Java worker to avoid undeclared Guava
[suztomo] Beam SQL JDBC driver not to declare unused Guava
[suztomo] KinesisIO to declare Guava dependency
[suztomo] ZetaSQL to declare Guava dependency
[suztomo] Removed unused dependency from elasticsearch-tests-2
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 1988284a89b10b60eea48325f8a3b370b551c77c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 1988284a89b10b60eea48325f8a3b370b551c77c
Commit message: "Merge pull request #10172: [BEAM-8747] Guava dependency cleanup"
> git rev-list --no-walk 6a921b879384b3f36d8d8447d82c7538699826aa # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4072336375483435601.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1996695873143654822.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2947712136249325666.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7522755005274497000.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5578560971383420514.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5222450603092699937.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/15/c8/b55cff970507d28cca487b644c738effbd4cbfb4065318f2e6b67f27944f/protobuf-3.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.11.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4798650433368290652.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-26 06:45:42,378 7e1af3f9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7e1af3f9/pkb.log>
2019-11-26 06:45:42,379 7e1af3f9 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-26 06:45:42,380 7e1af3f9 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-26 06:45:42,617 7e1af3f9 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-26 06:45:42,642 7e1af3f9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-26 06:45:42,724 7e1af3f9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-26 06:45:42,726 7e1af3f9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-26 06:45:42,727 7e1af3f9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-26 06:45:42,728 7e1af3f9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 06:45:42,728 7e1af3f9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-26 06:45:42,730 7e1af3f9 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 06:45:42,730 7e1af3f9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-26 06:45:42,730 7e1af3f9 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-26 06:45:42,730 7e1af3f9 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7e1af3f9/pkb.log>
2019-11-26 06:45:42,730 7e1af3f9 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7e1af3f9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#751
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/751/display/redirect?page=changes>
Changes:
[worldkzd] fix typos
[github] Update class_test.go
[worldkzd] keep 'a https'
[robertwb] [BEAM-8575] Added a unit test that Reshuffle preserves timestamps
[thw] [BEAM-8815] Skip manifest when no artifacts are staged
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6a921b879384b3f36d8d8447d82c7538699826aa (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6a921b879384b3f36d8d8447d82c7538699826aa
Commit message: "Merge pull request #10205: [BEAM-8815] Skip manifest when no artifacts are staged"
> git rev-list --no-walk 5fd93af49e6cb86ff52b20f103371df7e0447b7f # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8628204285113187119.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5615430844926404540.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4078110017988330161.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6494909278606160775.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6261019042424416541.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6307699663900060098.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1906855074530466705.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-26 01:08:26,854 c5e7d1ae MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/c5e7d1ae/pkb.log>
2019-11-26 01:08:26,854 c5e7d1ae MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-26 01:08:26,855 c5e7d1ae MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-26 01:08:27,434 c5e7d1ae MainThread INFO Setting --max_concurrent_threads=200.
2019-11-26 01:08:27,478 c5e7d1ae MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-26 01:08:27,533 c5e7d1ae MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-26 01:08:27,536 c5e7d1ae MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-26 01:08:27,536 c5e7d1ae MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-26 01:08:27,537 c5e7d1ae MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 01:08:27,537 c5e7d1ae MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-26 01:08:27,538 c5e7d1ae MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-26 01:08:27,539 c5e7d1ae MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-26 01:08:27,539 c5e7d1ae MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-26 01:08:27,539 c5e7d1ae MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/c5e7d1ae/pkb.log>
2019-11-26 01:08:27,539 c5e7d1ae MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/c5e7d1ae/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#750
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/750/display/redirect?page=changes>
Changes:
[kirillkozlov] Fix MongoDb SQL Integration Tests
[kirillkozlov] Add MongoDbIT back to build file
[kirillkozlov] Update JavaDoc comment and remove pipeline options
[pabloem] [BEAM-876] Support schemaUpdateOption in BigQueryIO (#9524)
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5fd93af49e6cb86ff52b20f103371df7e0447b7f (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 5fd93af49e6cb86ff52b20f103371df7e0447b7f
Commit message: "[BEAM-876] Support schemaUpdateOption in BigQueryIO (#9524)"
> git rev-list --no-walk 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins9155201032448085102.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3476476159593812496.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2901311150225040338.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8162863278467434850.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1654119981300172653.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6294748953230257590.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins9094737010728243013.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 18:45:52,786 0178872b MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/0178872b/pkb.log>
2019-11-25 18:45:52,787 0178872b MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 18:45:52,788 0178872b MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 18:45:53,276 0178872b MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 18:45:53,301 0178872b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 18:45:53,363 0178872b MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 18:45:53,366 0178872b MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 18:45:53,366 0178872b MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 18:45:53,367 0178872b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 18:45:53,368 0178872b MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 18:45:53,369 0178872b MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 18:45:53,369 0178872b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 18:45:53,370 0178872b MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 18:45:53,370 0178872b MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/0178872b/pkb.log>
2019-11-25 18:45:53,370 0178872b MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/0178872b/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#749
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/749/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 885ecbf3f49257e8b6b4ac376cb5a79ed6282580
Commit message: "[BEAM-8803] BigQuery Streaming Inserts are always retried by default. (#10195)"
> git rev-list --no-walk 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4622382529863336750.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2047585623690113538.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2040986794101956996.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins9147824693007884408.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.1)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5120467916042228519.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7871541180517434328.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.1)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6942982098688288456.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 12:45:40,703 9af69c50 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/9af69c50/pkb.log>
2019-11-25 12:45:40,703 9af69c50 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 12:45:40,704 9af69c50 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 12:45:41,218 9af69c50 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 12:45:41,242 9af69c50 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 12:45:41,298 9af69c50 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 12:45:41,301 9af69c50 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 12:45:41,301 9af69c50 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 12:45:41,302 9af69c50 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 12:45:41,302 9af69c50 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 12:45:41,303 9af69c50 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 12:45:41,304 9af69c50 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 12:45:41,304 9af69c50 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 12:45:41,304 9af69c50 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/9af69c50/pkb.log>
2019-11-25 12:45:41,304 9af69c50 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/9af69c50/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#748
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/748/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 885ecbf3f49257e8b6b4ac376cb5a79ed6282580
Commit message: "[BEAM-8803] BigQuery Streaming Inserts are always retried by default. (#10195)"
> git rev-list --no-walk 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins410685800097362381.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1274885681309449279.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins552092326889679043.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3413769885339018777.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1860121036043220711.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4649348422555548330.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8874661043030135329.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 06:45:44,676 bf923b62 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/bf923b62/pkb.log>
2019-11-25 06:45:44,677 bf923b62 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 06:45:44,678 bf923b62 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 06:45:45,395 bf923b62 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 06:45:45,422 bf923b62 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 06:45:45,511 bf923b62 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 06:45:45,513 bf923b62 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 06:45:45,514 bf923b62 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 06:45:45,515 bf923b62 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 06:45:45,516 bf923b62 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 06:45:45,517 bf923b62 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 06:45:45,518 bf923b62 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 06:45:45,518 bf923b62 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 06:45:45,518 bf923b62 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/bf923b62/pkb.log>
2019-11-25 06:45:45,518 bf923b62 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/bf923b62/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#747
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/747/display/redirect?page=changes>
Changes:
[github] [BEAM-8803] BigQuery Streaming Inserts are always retried by default.
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-9 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 885ecbf3f49257e8b6b4ac376cb5a79ed6282580 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 885ecbf3f49257e8b6b4ac376cb5a79ed6282580
Commit message: "[BEAM-8803] BigQuery Streaming Inserts are always retried by default. (#10195)"
> git rev-list --no-walk 639fc24b2a1d6886197750fe6be75e8ad8029205 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8511990102064467430.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4225643581959338989.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7632757416508376958.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4456412122344660161.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7988218661853095639.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6159279059944502168.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4889725342883177508.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-25 00:45:41,517 353906fc MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/353906fc/pkb.log>
2019-11-25 00:45:41,517 353906fc MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-25 00:45:41,519 353906fc MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-25 00:45:41,705 353906fc MainThread INFO Setting --max_concurrent_threads=200.
2019-11-25 00:45:41,735 353906fc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-25 00:45:41,798 353906fc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-25 00:45:41,801 353906fc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-25 00:45:41,801 353906fc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-25 00:45:41,802 353906fc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 00:45:41,803 353906fc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-25 00:45:41,804 353906fc MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-25 00:45:41,804 353906fc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-25 00:45:41,805 353906fc MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-25 00:45:41,805 353906fc MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/353906fc/pkb.log>
2019-11-25 00:45:41,805 353906fc MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/353906fc/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#746
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/746/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 639fc24b2a1d6886197750fe6be75e8ad8029205 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 639fc24b2a1d6886197750fe6be75e8ad8029205
Commit message: "Merge pull request #10021: [BEAM-8592] Adjusting ZetaSQL table resolution to standard"
> git rev-list --no-walk 639fc24b2a1d6886197750fe6be75e8ad8029205 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4826904850142049891.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4687098979915989129.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4049080310365071900.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1648350274030090548.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2002106289598149065.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3603864418288832702.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins678775726492106048.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 18:45:44,224 623df692 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/623df692/pkb.log>
2019-11-24 18:45:44,224 623df692 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 18:45:44,226 623df692 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 18:45:44,663 623df692 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 18:45:44,688 623df692 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 18:45:44,749 623df692 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 18:45:44,752 623df692 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 18:45:44,752 623df692 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 18:45:44,753 623df692 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 18:45:44,753 623df692 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 18:45:44,755 623df692 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 18:45:44,755 623df692 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 18:45:44,755 623df692 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 18:45:44,755 623df692 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/623df692/pkb.log>
2019-11-24 18:45:44,755 623df692 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/623df692/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#745
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/745/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 639fc24b2a1d6886197750fe6be75e8ad8029205 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 639fc24b2a1d6886197750fe6be75e8ad8029205
Commit message: "Merge pull request #10021: [BEAM-8592] Adjusting ZetaSQL table resolution to standard"
> git rev-list --no-walk 639fc24b2a1d6886197750fe6be75e8ad8029205 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2594560519924316550.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5892931546048557288.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5408171427931357661.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1120070239196582689.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7738981993157568886.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1998321091140408816.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3134178082626659676.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 12:46:00,301 eac6a3f7 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/eac6a3f7/pkb.log>
2019-11-24 12:46:00,301 eac6a3f7 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 12:46:00,303 eac6a3f7 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 12:46:00,846 eac6a3f7 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 12:46:00,870 eac6a3f7 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 12:46:00,932 eac6a3f7 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 12:46:00,935 eac6a3f7 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 12:46:00,935 eac6a3f7 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 12:46:00,936 eac6a3f7 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 12:46:00,937 eac6a3f7 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 12:46:00,938 eac6a3f7 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 12:46:00,938 eac6a3f7 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 12:46:00,938 eac6a3f7 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 12:46:00,938 eac6a3f7 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/eac6a3f7/pkb.log>
2019-11-24 12:46:00,938 eac6a3f7 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/eac6a3f7/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#744
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/744/display/redirect?page=changes>
Changes:
[kenn] Use only standard table resolution for ZetaSQL dialect
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-9 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 639fc24b2a1d6886197750fe6be75e8ad8029205 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 639fc24b2a1d6886197750fe6be75e8ad8029205
Commit message: "Merge pull request #10021: [BEAM-8592] Adjusting ZetaSQL table resolution to standard"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5474746434770540262.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1488574254235332980.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7785510710208396454.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins935842613794292282.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1215841378778331988.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins946659871832448465.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins317708997394693553.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 06:45:38,049 a51fd287 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/a51fd287/pkb.log>
2019-11-24 06:45:38,050 a51fd287 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 06:45:38,051 a51fd287 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 06:45:38,280 a51fd287 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 06:45:38,306 a51fd287 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 06:45:38,370 a51fd287 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 06:45:38,372 a51fd287 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 06:45:38,373 a51fd287 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 06:45:38,374 a51fd287 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 06:45:38,374 a51fd287 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 06:45:38,375 a51fd287 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 06:45:38,375 a51fd287 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 06:45:38,376 a51fd287 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 06:45:38,376 a51fd287 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/a51fd287/pkb.log>
2019-11-24 06:45:38,376 a51fd287 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/a51fd287/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#743
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/743/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins945926576931926352.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5195879140551709366.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4877950275308969676.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6314135698545309822.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (42.0.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1263336453019404658.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6301859811946675606.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (42.0.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3626952803721750699.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-24 00:45:40,461 d90bbb72 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d90bbb72/pkb.log>
2019-11-24 00:45:40,462 d90bbb72 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-24 00:45:40,463 d90bbb72 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-24 00:45:40,952 d90bbb72 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-24 00:45:40,976 d90bbb72 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-24 00:45:41,033 d90bbb72 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-24 00:45:41,035 d90bbb72 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-24 00:45:41,036 d90bbb72 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-24 00:45:41,037 d90bbb72 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 00:45:41,037 d90bbb72 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-24 00:45:41,038 d90bbb72 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-24 00:45:41,038 d90bbb72 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-24 00:45:41,039 d90bbb72 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-24 00:45:41,039 d90bbb72 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d90bbb72/pkb.log>
2019-11-24 00:45:41,039 d90bbb72 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d90bbb72/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#742
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/742/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1144400209595913434.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1730142338770716417.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins426979695688976934.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8770674897309579762.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2378275937836437953.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4547054347271301885.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1442325424628658283.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 18:45:44,975 db43eb00 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/db43eb00/pkb.log>
2019-11-23 18:45:44,976 db43eb00 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 18:45:44,977 db43eb00 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 18:45:45,503 db43eb00 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 18:45:45,528 db43eb00 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 18:45:45,592 db43eb00 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 18:45:45,595 db43eb00 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 18:45:45,595 db43eb00 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 18:45:45,596 db43eb00 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 18:45:45,597 db43eb00 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 18:45:45,598 db43eb00 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 18:45:45,599 db43eb00 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 18:45:45,599 db43eb00 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 18:45:45,599 db43eb00 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/db43eb00/pkb.log>
2019-11-23 18:45:45,599 db43eb00 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/db43eb00/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#741
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/741/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1399196699646392801.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5651144047707153812.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1867328227446582458.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1949970343071245162.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins807959661294006241.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins338274023415666042.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1059353494882958837.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 12:45:35,754 ea003041 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/ea003041/pkb.log>
2019-11-23 12:45:35,754 ea003041 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 12:45:35,755 ea003041 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 12:45:36,034 ea003041 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 12:45:36,057 ea003041 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 12:45:36,106 ea003041 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 12:45:36,109 ea003041 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 12:45:36,109 ea003041 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 12:45:36,110 ea003041 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 12:45:36,111 ea003041 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 12:45:36,112 ea003041 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 12:45:36,112 ea003041 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 12:45:36,112 ea003041 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 12:45:36,112 ea003041 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/ea003041/pkb.log>
2019-11-23 12:45:36,112 ea003041 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/ea003041/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#740
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/740/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk a42fd7492771505245b6dfa7063283593fc76993 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1630124060605448389.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3256905984390105762.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5710173168116908147.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2256422709724622454.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2633374892903719314.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7331377297122506185.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3613386961732117374.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 06:45:41,425 8eb18aca MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/8eb18aca/pkb.log>
2019-11-23 06:45:41,426 8eb18aca MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 06:45:41,427 8eb18aca MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 06:45:41,628 8eb18aca MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 06:45:41,652 8eb18aca MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 06:45:41,709 8eb18aca MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 06:45:41,712 8eb18aca MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 06:45:41,712 8eb18aca MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 06:45:41,713 8eb18aca MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 06:45:41,714 8eb18aca MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 06:45:41,728 8eb18aca MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 06:45:41,729 8eb18aca MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 06:45:41,729 8eb18aca MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 06:45:41,729 8eb18aca MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/8eb18aca/pkb.log>
2019-11-23 06:45:41,729 8eb18aca MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/8eb18aca/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#739
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/739/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-8805] Remove obsolete worker_threads experiment in tests
[sunjincheng121] [BEAM-8619] Move reusable information to BundleProcessor.
[sunjincheng121] [BEAM-8619] Extract tearDown functions into BundleProcessor.
[sunjincheng121] [BEAM-8619] Reuse the BundleProcessor between bundles for the same
[sunjincheng121] [BEAM-8619] Teardown the DoFns when upon control service termination for
[tweise] Documenting how to use Beam twitter handle (#10082)
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a42fd7492771505245b6dfa7063283593fc76993 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a42fd7492771505245b6dfa7063283593fc76993
Commit message: "Documenting how to use Beam twitter handle (#10082)"
> git rev-list --no-walk 031a8c34b218b8bb5b124b1d95af93c19a185516 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3405685096971407873.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8892118582024805129.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6284384099846825401.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3278481972129435292.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5683342936788468408.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4716834265735174493.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4918223616900349185.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-23 00:57:52,096 af36cb94 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/af36cb94/pkb.log>
2019-11-23 00:57:52,096 af36cb94 MainThread INFO PerfKitBenchmarker version: v1.12.0-1544-gbded197
2019-11-23 00:57:52,097 af36cb94 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-23 00:57:52,501 af36cb94 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-23 00:57:52,526 af36cb94 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-23 00:57:52,584 af36cb94 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-23 00:57:52,604 af36cb94 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-23 00:57:52,605 af36cb94 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-23 00:57:52,606 af36cb94 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 00:57:52,606 af36cb94 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-23 00:57:52,624 af36cb94 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-23 00:57:52,625 af36cb94 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-23 00:57:52,625 af36cb94 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-23 00:57:52,625 af36cb94 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/af36cb94/pkb.log>
2019-11-23 00:57:52,625 af36cb94 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/af36cb94/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#738
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/738/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-8658] [BEAM-8781] Optionally set jar and artifact staging port in
[kcweaver] Pass artifact port to FlinkJarJobServer as well.
[kcweaver] Move FlinkRunnerOptions to pipeline_options.
[kcweaver] [BEAM-8796] Optionally configure static ports for job and expansion
[robertwb] [BEAM-8802] Don't clear watermark hold when adding elements.
[lukecwik] [BEAM-7948] Add time-based cache threshold support in the Java data s…
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-11 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 031a8c34b218b8bb5b124b1d95af93c19a185516 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 031a8c34b218b8bb5b124b1d95af93c19a185516
Commit message: "Merge pull request #10191 [BEAM-8802] Don't clear watermark hold when adding elements."
> git rev-list --no-walk 07d952f313477ee18cdc706100ba7e1810b1ef4f # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins189113542653557407.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2086212794231497081.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6805402069148163216.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins128870751329036871.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8529762273794195637.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1498428323632281593.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2891301316874805907.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 19:29:23,511 0502ae38 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/0502ae38/pkb.log>
2019-11-22 19:29:23,511 0502ae38 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 19:29:23,512 0502ae38 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 19:29:23,963 0502ae38 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 19:29:23,988 0502ae38 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 19:29:24,055 0502ae38 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 19:29:24,058 0502ae38 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 19:29:24,058 0502ae38 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 19:29:24,059 0502ae38 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 19:29:24,060 0502ae38 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 19:29:24,061 0502ae38 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 19:29:24,061 0502ae38 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 19:29:24,062 0502ae38 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 19:29:24,062 0502ae38 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/0502ae38/pkb.log>
2019-11-22 19:29:24,062 0502ae38 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/0502ae38/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#737
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/737/display/redirect?page=changes>
Changes:
[33895511+aromanenko-dev] [BEAM-7636] Migrate SqsIO to AWS SDK V2 for Java
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-9 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 07d952f313477ee18cdc706100ba7e1810b1ef4f (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 07d952f313477ee18cdc706100ba7e1810b1ef4f
Commit message: "[BEAM-7636] Migrate SqsIO to AWS SDK V2 for Java"
> git rev-list --no-walk 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1039770762147097488.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5874858686885051798.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins574667585271519262.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins657848602381301741.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3795424159089584139.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins209406200262650014.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8891454793529839649.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 12:45:46,996 cb0a4b81 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/cb0a4b81/pkb.log>
2019-11-22 12:45:46,997 cb0a4b81 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 12:45:46,998 cb0a4b81 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 12:45:47,473 cb0a4b81 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 12:45:47,498 cb0a4b81 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 12:45:47,560 cb0a4b81 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 12:45:47,562 cb0a4b81 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 12:45:47,563 cb0a4b81 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 12:45:47,564 cb0a4b81 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 12:45:47,564 cb0a4b81 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 12:45:47,565 cb0a4b81 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 12:45:47,566 cb0a4b81 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 12:45:47,566 cb0a4b81 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 12:45:47,566 cb0a4b81 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/cb0a4b81/pkb.log>
2019-11-22 12:45:47,566 cb0a4b81 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/cb0a4b81/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#736
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/736/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb
Commit message: "[BEAM-7278, BEAM-2530] Add support for using a Java linkage testing tool to aid upgrading dependencies. (#10184)"
> git rev-list --no-walk 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1844495353982334502.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6569933532274798837.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2086009655371078665.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7134941333625151546.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins805514670104007822.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2170281902869168482.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5310404884373787731.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 06:45:39,616 7d9df43d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7d9df43d/pkb.log>
2019-11-22 06:45:39,616 7d9df43d MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 06:45:39,618 7d9df43d MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 06:45:40,087 7d9df43d MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 06:45:40,112 7d9df43d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 06:45:40,180 7d9df43d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 06:45:40,183 7d9df43d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 06:45:40,183 7d9df43d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 06:45:40,184 7d9df43d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 06:45:40,185 7d9df43d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 06:45:40,186 7d9df43d MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 06:45:40,186 7d9df43d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 06:45:40,187 7d9df43d MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 06:45:40,187 7d9df43d MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7d9df43d/pkb.log>
2019-11-22 06:45:40,187 7d9df43d MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7d9df43d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#735
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/735/display/redirect?page=changes>
Changes:
[github] [BEAM-8659] RowJsonTest should test serialization independently
[github] [BEAM-8743] Add support for flat schemas in pubsub
[pabloem] [BEAM-8016] Pipeline Graph (#10132)
[github] [BEAM-7278, BEAM-2530] Add support for using a Java linkage testing tool
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7a8a26b6f1e67c619bfe283492a3f9fe83a983bb
Commit message: "[BEAM-7278, BEAM-2530] Add support for using a Java linkage testing tool to aid upgrading dependencies. (#10184)"
> git rev-list --no-walk 07a5155974a72b336d686efc77cc8a6dd71aa581 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2985538746875718718.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4523320405413787235.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5006838972031148118.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3251601839675653858.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins815138318556429486.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5566407117615214133.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6733404152084415132.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-22 01:17:12,244 90495b27 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/90495b27/pkb.log>
2019-11-22 01:17:12,245 90495b27 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-22 01:17:12,246 90495b27 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-22 01:17:12,430 90495b27 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-22 01:17:12,455 90495b27 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-22 01:17:12,516 90495b27 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-22 01:17:12,518 90495b27 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-22 01:17:12,519 90495b27 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-22 01:17:12,520 90495b27 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 01:17:12,520 90495b27 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-22 01:17:12,522 90495b27 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-22 01:17:12,522 90495b27 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-22 01:17:12,522 90495b27 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-22 01:17:12,522 90495b27 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/90495b27/pkb.log>
2019-11-22 01:17:12,522 90495b27 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/90495b27/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#734
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/734/display/redirect?page=changes>
Changes:
[bhulette] Fix inverted timeout check in TestPubsub
[bhulette] Add empty check before acknowledge
[echauchot] [BEAM-8470] Enable custom window tests in new spark runner validates
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 07a5155974a72b336d686efc77cc8a6dd71aa581 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 07a5155974a72b336d686efc77cc8a6dd71aa581
Commit message: "Merge pull request #10153: [BEAM-8740] TestPubsub ignores timeout"
> git rev-list --no-walk cd36c75174ed0f1be806b8aa7a4819bacbededb7 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7121049392528393745.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6454083683366813331.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5248404155718685673.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2450092661036539882.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6864498798194448797.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3117204685974322546.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2257518278050667602.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 19:01:25,207 7e7ee251 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7e7ee251/pkb.log>
2019-11-21 19:01:25,207 7e7ee251 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 19:01:25,209 7e7ee251 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 19:01:25,557 7e7ee251 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 19:01:25,582 7e7ee251 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 19:01:25,647 7e7ee251 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 19:01:25,650 7e7ee251 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 19:01:25,650 7e7ee251 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 19:01:25,652 7e7ee251 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 19:01:25,652 7e7ee251 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 19:01:25,653 7e7ee251 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 19:01:25,654 7e7ee251 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 19:01:25,654 7e7ee251 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 19:01:25,654 7e7ee251 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7e7ee251/pkb.log>
2019-11-21 19:01:25,654 7e7ee251 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/7e7ee251/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#733
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/733/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision cd36c75174ed0f1be806b8aa7a4819bacbededb7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f cd36c75174ed0f1be806b8aa7a4819bacbededb7
Commit message: "Merge pull request #10181: Website: "common" --> "unique" (and other word choice fixes)"
> git rev-list --no-walk cd36c75174ed0f1be806b8aa7a4819bacbededb7 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6255080805541548641.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3490883219331036687.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6617721135926578394.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8004944422743634810.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6024361295365146585.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3472942777207442273.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3247918200975222327.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 12:45:41,277 3e6eaee4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/3e6eaee4/pkb.log>
2019-11-21 12:45:41,278 3e6eaee4 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 12:45:41,279 3e6eaee4 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 12:45:41,772 3e6eaee4 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 12:45:41,798 3e6eaee4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 12:45:41,868 3e6eaee4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 12:45:41,871 3e6eaee4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 12:45:41,872 3e6eaee4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 12:45:41,874 3e6eaee4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 12:45:41,874 3e6eaee4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 12:45:41,875 3e6eaee4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 12:45:41,876 3e6eaee4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 12:45:41,876 3e6eaee4 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 12:45:41,876 3e6eaee4 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/3e6eaee4/pkb.log>
2019-11-21 12:45:41,876 3e6eaee4 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/3e6eaee4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#732
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/732/display/redirect?page=changes>
Changes:
[github] common --> unique
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision cd36c75174ed0f1be806b8aa7a4819bacbededb7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f cd36c75174ed0f1be806b8aa7a4819bacbededb7
Commit message: "Merge pull request #10181: Website: "common" --> "unique" (and other word choice fixes)"
> git rev-list --no-walk 5e3adf5e25af372253da42a28e8318b6f535c063 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8572635599635292935.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins509043789577163259.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8756069659192185870.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3719598536069045352.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7231895376657883887.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4391375899050351542.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins4563048550807810148.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 06:45:37,397 1622444a MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/1622444a/pkb.log>
2019-11-21 06:45:37,397 1622444a MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 06:45:37,399 1622444a MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 06:45:37,601 1622444a MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 06:45:37,626 1622444a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 06:45:37,690 1622444a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 06:45:37,692 1622444a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 06:45:37,693 1622444a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 06:45:37,694 1622444a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 06:45:37,694 1622444a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 06:45:37,696 1622444a MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 06:45:37,696 1622444a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 06:45:37,696 1622444a MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 06:45:37,696 1622444a MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/1622444a/pkb.log>
2019-11-21 06:45:37,696 1622444a MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/1622444a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#731
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/731/display/redirect?page=changes>
Changes:
[M.Yzvenn] fix - 1MB is interpreted as 1000, not 1024
[robertwb] [BEAM-8629] Don't return mutable class type hints.
[valentyn] Guard pickling operations with a lock to prevent race condition in
[kcweaver] [BEAM-8795] fix Spark runner build
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5e3adf5e25af372253da42a28e8318b6f535c063 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 5e3adf5e25af372253da42a28e8318b6f535c063
Commit message: "Merge pull request #10167 [BEAM-8651] Guard pickling operations with a lock to prevent race condition in module imports."
> git rev-list --no-walk 940757816402f198404e3c26c110a6d0e7c81f47 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1112515437793473105.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3017467889751562559.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins343398600382006457.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins7762960508808332564.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6855515628538246868.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2418708186330900569.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Downloading https://files.pythonhosted.org/packages/e9/17/5a26eb5fdbf9f78afc1438501eceb418b3c73eb36241e4da19bc37a9922c/google_cloud_monitoring-0.34.0-py2.py3-none-any.whl (220kB)
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.34.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins2992201811371408946.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-21 01:20:46,328 d2abf282 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d2abf282/pkb.log>
2019-11-21 01:20:46,328 d2abf282 MainThread INFO PerfKitBenchmarker version: v1.12.0-1538-g57a463e
2019-11-21 01:20:46,329 d2abf282 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-21 01:20:46,770 d2abf282 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-21 01:20:46,795 d2abf282 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-21 01:20:46,854 d2abf282 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-21 01:20:46,857 d2abf282 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-21 01:20:46,857 d2abf282 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-21 01:20:46,858 d2abf282 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 01:20:46,858 d2abf282 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-21 01:20:46,860 d2abf282 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-21 01:20:46,860 d2abf282 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-21 01:20:46,860 d2abf282 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-21 01:20:46,860 d2abf282 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d2abf282/pkb.log>
2019-11-21 01:20:46,860 d2abf282 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d2abf282/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_WordCountIT_Py37
#730
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/730/display/redirect?page=changes>
Changes:
[katarzyna.kucharczyk] [BEAM-6335] Added streaming GroupByKey Test that reads SyntheticSource
[katarzyna.kucharczyk] [BEAM-6335] Changed SyntheticDataPublisher to publish String UTF values
[katarzyna.kucharczyk] [BEAM-6335] Added custom PubSub Matcher that stops pipeline after
[valentyn] Restore original behavior of evaluating worker host on Windows until a
[echauchot] [BEAM-8470] Add an empty spark-structured-streaming runner project
[echauchot] [BEAM-8470] Fix missing dep
[echauchot] [BEAM-8470] Add SparkPipelineOptions
[echauchot] [BEAM-8470] Start pipeline translation
[echauchot] [BEAM-8470] Add global pipeline translation structure
[echauchot] [BEAM-8470] Add nodes translators structure
[echauchot] [BEAM-8470] Wire node translators with pipeline translator
[echauchot] [BEAM-8470] Renames: better differenciate pipeline translator for
[echauchot] [BEAM-8470] Organise methods in PipelineTranslator
[echauchot] [BEAM-8470] Initialise BatchTranslationContext
[echauchot] [BEAM-8470] Refactoring: -move batch/streaming common translation
[echauchot] [BEAM-8470] Make transform translation clearer: renaming, comments
[echauchot] [BEAM-8470] Improve javadocs
[echauchot] [BEAM-8470] Move SparkTransformOverrides to correct package
[echauchot] [BEAM-8470] Move common translation context components to superclass
[echauchot] [BEAM-8470] apply spotless
[echauchot] [BEAM-8470] Make codestyle and firebug happy
[echauchot] [BEAM-8470] Add TODOs
[echauchot] [BEAM-8470] Post-pone batch qualifier in all classes names for
[echauchot] [BEAM-8470] Add precise TODO for multiple TransformTranslator per
[echauchot] [BEAM-8470] Added SparkRunnerRegistrar
[echauchot] [BEAM-8470] Add basic pipeline execution. Refactor translatePipeline()
[echauchot] [BEAM-8470] Create PCollections manipulation methods
[echauchot] [BEAM-8470] Create Datasets manipulation methods
[echauchot] [BEAM-8470] Add Flatten transformation translator
[echauchot] [BEAM-8470] Add primitive GroupByKeyTranslatorBatch implementation
[echauchot] [BEAM-8470] Use Iterators.transform() to return Iterable
[echauchot] [BEAM-8470] Implement read transform
[echauchot] [BEAM-8470] update TODO
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] start source instanciation
[echauchot] [BEAM-8470] Improve exception flow
[echauchot] [BEAM-8470] Improve type enforcement in ReadSourceTranslator
[echauchot] [BEAM-8470] Experiment over using spark Catalog to pass in Beam Source
[echauchot] [BEAM-8470] Add source mocks
[echauchot] [BEAM-8470] fix mock, wire mock in translators and create a main test.
[echauchot] [BEAM-8470] Use raw WindowedValue so that spark Encoders could work
[echauchot] [BEAM-8470] clean deps
[echauchot] [BEAM-8470] Move DatasetSourceMock to proper batch mode
[echauchot] [BEAM-8470] Run pipeline in batch mode or in streaming mode
[echauchot] [BEAM-8470] Split batch and streaming sources and translators
[echauchot] [BEAM-8470] Use raw Encoder<WindowedValue> also in regular
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Add ReadSourceTranslatorStreaming
[echauchot] [BEAM-8470] Move Source and translator mocks to a mock package.
[echauchot] [BEAM-8470] Pass Beam Source and PipelineOptions to the spark DataSource
[echauchot] [BEAM-8470] Refactor DatasetSource fields
[echauchot] [BEAM-8470] Wire real SourceTransform and not mock and update the test
[echauchot] [BEAM-8470] Add missing 0-arg public constructor
[echauchot] [BEAM-8470] Use new PipelineOptionsSerializationUtils
[echauchot] [BEAM-8470] Apply spotless and fix checkstyle
[echauchot] [BEAM-8470] Add a dummy schema for reader
[echauchot] [BEAM-8470] Add empty 0-arg constructor for mock source
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Checkstyle and Findbugs
[echauchot] [BEAM-8470] Refactor SourceTest to a UTest instaed of a main
[echauchot] [BEAM-8470] Fix pipeline triggering: use a spark action instead of
[echauchot] [BEAM-8470] improve readability of options passing to the source
[echauchot] [BEAM-8470] Clean unneeded fields in DatasetReader
[echauchot] [BEAM-8470] Fix serialization issues
[echauchot] [BEAM-8470] Add SerializationDebugger
[echauchot] [BEAM-8470] Add serialization test
[echauchot] [BEAM-8470] Move SourceTest to same package as tested class
[echauchot] [BEAM-8470] Fix SourceTest
[echauchot] [BEAM-8470] Simplify beam reader creation as it created once the source
[echauchot] [BEAM-8470] Put all transform translators Serializable
[echauchot] [BEAM-8470] Enable test mode
[echauchot] [BEAM-8470] Enable gradle build scan
[echauchot] [BEAM-8470] Add flatten test
[echauchot] [BEAM-8470] First attempt for ParDo primitive implementation
[echauchot] [BEAM-8470] Serialize windowedValue to byte[] in source to be able to
[echauchot] [BEAM-8470] Comment schema choices
[echauchot] [BEAM-8470] Fix errorprone
[echauchot] [BEAM-8470] Fix testMode output to comply with new binary schema
[echauchot] [BEAM-8470] Cleaning
[echauchot] [BEAM-8470] Remove bundleSize parameter and always use spark default
[echauchot] [BEAM-8470] Fix split bug
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Add ParDoTest
[echauchot] [BEAM-8470] Address minor review notes
[echauchot] [BEAM-8470] Clean
[echauchot] [BEAM-8470] Add GroupByKeyTest
[echauchot] [BEAM-8470] Add comments and TODO to GroupByKeyTranslatorBatch
[echauchot] [BEAM-8470] Fix type checking with Encoder of WindowedValue<T>
[echauchot] [BEAM-8470] Port latest changes of ReadSourceTranslatorBatch to
[echauchot] [BEAM-8470] Remove no more needed putDatasetRaw
[echauchot] [BEAM-8470] Add ComplexSourceTest
[echauchot] [BEAM-8470] Fail in case of having SideInouts or State/Timers
[echauchot] [BEAM-8470] Fix Encoders: create an Encoder for every manipulated type
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] Fixed Javadoc error
[echauchot] [BEAM-8470] Rename SparkSideInputReader class and rename pruneOutput()
[echauchot] [BEAM-8470] Don't use deprecated
[echauchot] [BEAM-8470] Simplify logic of ParDo translator
[echauchot] [BEAM-8470] Fix kryo issue in GBK translator with a workaround
[echauchot] [BEAM-8470] Rename SparkOutputManager for consistency
[echauchot] [BEAM-8470] Fix for test elements container in GroupByKeyTest
[echauchot] [BEAM-8470] Added "testTwoPardoInRow"
[echauchot] [BEAM-8470] Add a test for the most simple possible Combine
[echauchot] [BEAM-8470] Rename SparkDoFnFilterFunction to DoFnFilterFunction for
[echauchot] [BEAM-8470] Generalize the use of SerializablePipelineOptions in place
[echauchot] [BEAM-8470] Fix getSideInputs
[echauchot] [BEAM-8470] Extract binary schema creation in a helper class
[echauchot] [BEAM-8470] First version of combinePerKey
[echauchot] [BEAM-8470] Improve type checking of Tuple2 encoder
[echauchot] [BEAM-8470] Introduce WindowingHelpers (and helpers package) and use it
[echauchot] [BEAM-8470] Fix combiner using KV as input, use binary encoders in place
[echauchot] [BEAM-8470] Add combinePerKey and CombineGlobally tests
[echauchot] [BEAM-8470] Introduce RowHelpers
[echauchot] [BEAM-8470] Add CombineGlobally translation to avoid translating
[echauchot] [BEAM-8470] Cleaning
[echauchot] [BEAM-8470] Get back to classes in translators resolution because URNs
[echauchot] [BEAM-8470] Fix various type checking issues in Combine.Globally
[echauchot] [BEAM-8470] Update test with Long
[echauchot] [BEAM-8470] Fix combine. For unknown reason GenericRowWithSchema is used
[echauchot] [BEAM-8470] Use more generic Row instead of GenericRowWithSchema
[echauchot] [BEAM-8470] Add explanation about receiving a Row as input in the
[echauchot] [BEAM-8470] Fix encoder bug in combinePerkey
[echauchot] [BEAM-8470] Cleaning
[echauchot] [BEAM-8470] Implement WindowAssignTranslatorBatch
[echauchot] [BEAM-8470] Implement WindowAssignTest
[echauchot] [BEAM-8470] Fix javadoc
[echauchot] [BEAM-8470] Added SideInput support
[echauchot] [BEAM-8470] Fix CheckStyle violations
[echauchot] [BEAM-8470] Don't use Reshuffle translation
[echauchot] [BEAM-8470] Added using CachedSideInputReader
[echauchot] [BEAM-8470] Added TODO comment for ReshuffleTranslatorBatch
[echauchot] [BEAM-8470] And unchecked warning suppression
[echauchot] [BEAM-8470] Add streaming source initialisation
[echauchot] [BEAM-8470] Implement first streaming source
[echauchot] [BEAM-8470] Add a TODO on spark output modes
[echauchot] [BEAM-8470] Add transformators registry in PipelineTranslatorStreaming
[echauchot] [BEAM-8470] Add source streaming test
[echauchot] [BEAM-8470] Specify checkpointLocation at the pipeline start
[echauchot] [BEAM-8470] Clean unneeded 0 arg constructor in batch source
[echauchot] [BEAM-8470] Clean streaming source
[echauchot] [BEAM-8470] Continue impl of offsets for streaming source
[echauchot] [BEAM-8470] Deal with checkpoint and offset based read
[echauchot] [BEAM-8470] Apply spotless and fix spotbugs warnings
[echauchot] [BEAM-8470] Disable never ending test
[echauchot] [BEAM-8470] Fix access level issues, typos and modernize code to Java 8
[echauchot] [BEAM-8470] Merge Spark Structured Streaming runner into main Spark
[echauchot] [BEAM-8470] Fix non-vendored imports from Spark Streaming Runner classes
[echauchot] [BEAM-8470] Pass doFnSchemaInformation to ParDo batch translation
[echauchot] [BEAM-8470] Fix spotless issues after rebase
[echauchot] [BEAM-8470] Fix logging levels in Spark Structured Streaming translation
[echauchot] [BEAM-8470] Add SparkStructuredStreamingPipelineOptions and
[echauchot] [BEAM-8470] Rename SparkPipelineResult to
[echauchot] [BEAM-8470] Use PAssert in Spark Structured Streaming transform tests
[echauchot] [BEAM-8470] Ignore spark offsets (cf javadoc)
[echauchot] [BEAM-8470] implement source.stop
[echauchot] [BEAM-8470] Update javadoc
[echauchot] [BEAM-8470] Apply Spotless
[echauchot] [BEAM-8470] Enable batch Validates Runner tests for Structured Streaming
[echauchot] [BEAM-8470] Limit the number of partitions to make tests go 300% faster
[echauchot] [BEAM-8470] Fixes ParDo not calling setup and not tearing down if
[echauchot] [BEAM-8470] Pass transform based doFnSchemaInformation in ParDo
[echauchot] [BEAM-8470] Consider null object case on RowHelpers, fixes empty side
[echauchot] [BEAM-8470] Put back batch/simpleSourceTest.testBoundedSource
[echauchot] [BEAM-8470] Update windowAssignTest
[echauchot] [BEAM-8470] Add comment about checkpoint mark
[echauchot] [BEAM-8470] Re-code GroupByKeyTranslatorBatch to conserve windowing
[echauchot] [BEAM-8470] re-enable reduceFnRunner timers for output
[echauchot] [BEAM-8470] Improve visibility of debug messages
[echauchot] [BEAM-8470] Add a test that GBK preserves windowing
[echauchot] [BEAM-8470] Add TODO in Combine translations
[echauchot] [BEAM-8470] Update KVHelpers.extractKey() to deal with WindowedValue and
[echauchot] [BEAM-8470] Fix comment about schemas
[echauchot] [BEAM-8470] Implement reduce part of CombineGlobally translation with
[echauchot] [BEAM-8470] Output data after combine
[echauchot] [BEAM-8470] Implement merge accumulators part of CombineGlobally
[echauchot] [BEAM-8470] Fix encoder in combine call
[echauchot] [BEAM-8470] Revert extractKey while combinePerKey is not done (so that
[echauchot] [BEAM-8470] Apply a groupByKey avoids for some reason that the spark
[echauchot] [BEAM-8470] Fix case when a window does not merge into any other window
[echauchot] [BEAM-8470] Fix wrong encoder in combineGlobally GBK
[echauchot] [BEAM-8470] Fix bug in the window merging logic
[echauchot] [BEAM-8470] Remove the mapPartition that adds a key per partition
[echauchot] [BEAM-8470] Remove CombineGlobally translation because it is less
[echauchot] [BEAM-8470] Now that there is only Combine.PerKey translation, make only
[echauchot] [BEAM-8470] Clean no more needed KVHelpers
[echauchot] [BEAM-8470] Clean not more needed RowHelpers
[echauchot] [BEAM-8470] Clean not more needed WindowingHelpers
[echauchot] [BEAM-8470] Fix javadoc of AggregatorCombiner
[echauchot] [BEAM-8470] Fixed immutable list bug
[echauchot] [BEAM-8470] add comment in combine globally test
[echauchot] [BEAM-8470] Clean groupByKeyTest
[echauchot] [BEAM-8470] Add a test that combine per key preserves windowing
[echauchot] [BEAM-8470] Ignore for now not working test testCombineGlobally
[echauchot] [BEAM-8470] Add metrics support in DoFn
[echauchot] [BEAM-8470] Add missing dependencies to run Spark Structured Streaming
[echauchot] [BEAM-8470] Add setEnableSparkMetricSinks() method
[echauchot] [BEAM-8470] Fix javadoc
[echauchot] [BEAM-8470] Fix accumulators initialization in Combine that prevented
[echauchot] [BEAM-8470] Add a test to check that CombineGlobally preserves windowing
[echauchot] [BEAM-8470] Persist all output Dataset if there are multiple outputs in
[echauchot] [BEAM-8470] Added metrics sinks and tests
[echauchot] [BEAM-8470] Make spotless happy
[echauchot] [BEAM-8470] Add PipelineResults to Spark structured streaming.
[echauchot] [BEAM-8470] Update log4j configuration
[echauchot] [BEAM-8470] Add spark execution plans extended debug messages.
[echauchot] [BEAM-8470] Print number of leaf datasets
[echauchot] [BEAM-8470] fixup! Add PipelineResults to Spark structured streaming.
[echauchot] [BEAM-8470] Remove no more needed AggregatorCombinerPerKey (there is
[echauchot] [BEAM-8470] After testing performance and correctness, launch pipeline
[echauchot] [BEAM-8470] Improve Pardo translation performance: avoid calling a
[echauchot] [BEAM-8470] Use "sparkMaster" in local mode to obtain number of shuffle
[echauchot] [BEAM-8470] Wrap Beam Coders into Spark Encoders using
[echauchot] [BEAM-8470] Wrap Beam Coders into Spark Encoders using
[echauchot] [BEAM-8470] type erasure: spark encoders require a Class<T>, pass Object
[echauchot] [BEAM-8470] Fix scala Product in Encoders to avoid StackEverflow
[echauchot] [BEAM-8470] Conform to spark ExpressionEncoders: pass classTags,
[echauchot] [BEAM-8470] Add a simple spark native test to test Beam coders wrapping
[echauchot] [BEAM-8470] Fix code generation in Beam coder wrapper
[echauchot] [BEAM-8470] Lazy init coder because coder instance cannot be
[echauchot] [BEAM-8470] Fix warning in coder construction by reflexion
[echauchot] [BEAM-8470] Fix ExpressionEncoder generated code: typos, try catch, fqcn
[echauchot] [BEAM-8470] Fix getting the output value in code generation
[echauchot] [BEAM-8470] Fix beam coder lazy init using reflexion: use .clas + try
[echauchot] [BEAM-8470] Remove lazy init of beam coder because there is no generic
[echauchot] [BEAM-8470] Remove example code
[echauchot] [BEAM-8470] Fix equal and hashcode
[echauchot] [BEAM-8470] Fix generated code: uniform exceptions catching, fix
[echauchot] [BEAM-8470] Add an assert of equality in the encoders test
[echauchot] [BEAM-8470] Apply spotless and checkstyle and add javadocs
[echauchot] [BEAM-8470] Wrap exceptions in UserCoderExceptions
[echauchot] [BEAM-8470] Put Encoders expressions serializable
[echauchot] [BEAM-8470] Catch Exception instead of IOException because some coders
[echauchot] [BEAM-8470] Apply new Encoders to CombinePerKey
[echauchot] [BEAM-8470] Apply new Encoders to Read source
[echauchot] [BEAM-8470] Improve performance of source: the mapper already calls
[echauchot] [BEAM-8470] Ignore long time failing test: SparkMetricsSinkTest
[echauchot] [BEAM-8470] Apply new Encoders to Window assign translation
[echauchot] [BEAM-8470] Apply new Encoders to AggregatorCombiner
[echauchot] [BEAM-8470] Create a Tuple2Coder to encode scala tuple2
[echauchot] [BEAM-8470] Apply new Encoders to GroupByKey
[echauchot] [BEAM-8470] Apply new Encoders to Pardo. Replace Tuple2Coder with
[echauchot] [BEAM-8470] Apply spotless, fix typo and javadoc
[echauchot] [BEAM-8470] Use beam encoders also in the output of the source
[echauchot] [BEAM-8470] Remove unneeded cast
[echauchot] [BEAM-8470] Fix: Remove generic hack of using object. Use actual Coder
[echauchot] [BEAM-8470] Remove Encoders based on kryo now that we call Beam coders
[echauchot] [BEAM-8470] Add a jenkins job for validates runner tests in the new
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] Rebase on master: pass sideInputMapping in SimpleDoFnRunner
[echauchot] Fix SpotBugs
[echauchot] [BEAM-8470] simplify coders in combinePerKey translation
[echauchot] [BEAM-8470] Fix combiner. Do not reuse instance of accumulator
[echauchot] [BEAM-8470] input windows can arrive exploded (for sliding windows). As
[echauchot] [BEAM-8470] Add a combine test with sliding windows
[echauchot] [BEAM-8470] Add a test to test combine translation on binaryCombineFn
[echauchot] [BEAM-8470] Fix tests: use correct
[echauchot] [BEAM-8470] Fix wrong expected results in
[echauchot] [BEAM-8470] Add disclaimers about this runner being experimental
[echauchot] [BEAM-8470] Fix: create an empty accumulator in
[echauchot] [BEAM-8470] Apply spotless
[echauchot] [BEAM-8470] Add a countPerElement test with sliding windows
[echauchot] [BEAM-8470] Fix the output timestamps of combine: timestamps must be
[echauchot] [BEAM-8470] set log level to info to avoid resource consumption in
[echauchot] [BEAM-8470] Fix CombineTest.testCountPerElementWithSlidingWindows
[aromanenko.dev] [BEAM-8470] Remove "validatesStructuredStreamingRunnerBatch" from
[echauchot] [BEAM-8470] Fix timestamps in combine output: assign the timestamp to
[iemejia] [website] Add Spark Structured Runner VR badge to the github template
[tvalentyn] [BEAM-8575] Add a Python test to test windowing in DoFn finish_bundle()
[github] [BEAM-3419] Flesh out iterable side inputs and key enumeration for
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 940757816402f198404e3c26c110a6d0e7c81f47 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 940757816402f198404e3c26c110a6d0e7c81f47
Commit message: "[BEAM-3419] Flesh out iterable side inputs and key enumeration for multimaps in shared libraries (#10147)"
> git rev-list --no-walk 0b415fd7e9dd5c80034ca237e08c3959ec78ffe3 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins3814997289372913521.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins1798320928862987204.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8575849767602178040.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env> --python=python2.7
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python2.7>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python>
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python2.7
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins8160693021770489125.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.6.0)
Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.3.1)
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins5426914967966228522.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker>
Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker'...>
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins363544631188515436.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt>
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/jenkins/.cache/pip/wheels/a7/15/a0/0a0561549ad11cdc1bc8fa1191a353efd30facf6bfb507aefc/absl_py-0.8.1-cp27-none-any.whl
Collecting jinja2>=2.7
Using cached https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.6.0)
Collecting colorlog[windows]==2.6.0
Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/92/a0/00/8690a57883956a301d91cf4ec999cc0b258b01e3f548f86e89/blinker-1.4-cp27-none-any.whl
Collecting futures>=3.0.3
Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f/PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl
Collecting pint>=0.7
Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl
Collecting numpy==1.16.4
Using cached https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl
Processing /home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Collecting contextlib2>=0.5.1
Using cached https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578/pywinrm-0.4.1-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f1/e6/ea/7387e3629cb46ba65140141f972745b823f4486c6fe884ccb8/timeout_decorator-0.4.1-cp27-none-any.whl
Collecting google-cloud-monitoring
Using cached https://files.pythonhosted.org/packages/0d/24/35da9c690c1d5042d45c40bbce49daf65e8a754b38f78f8c3d394bd12843/google_cloud_monitoring-0.33.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting six
Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting colorama; extra == "windows"
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting funcsigs; python_version == "2.7"
Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Collecting requests-ntlm>=0.3.0
Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Collecting requests>=2.9.1
Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting xmltodict
Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
Using cached https://files.pythonhosted.org/packages/29/3a/c528ef37f48d6ffba16f0f3c0426456ba21e0dd32be9c61a2ade93e07faa/google_api_core-1.14.3-py2.py3-none-any.whl
Collecting cryptography>=1.3
Using cached https://files.pythonhosted.org/packages/c8/52/ad7f2cbe3b6e9340526dc401b38bb67c06160449446e15b0bf5f947fa168/cryptography-2.8-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting ntlm-auth>=1.0.2
Using cached https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/9e/3d/a2/1bec8bb7db80ab3216dbc33092bb7ccd0debfb8ba42b5668d5/googleapis_common_protos-1.6.0-cp27-none-any.whl
Collecting protobuf>=3.4.0
Using cached https://files.pythonhosted.org/packages/c5/49/ffa7ab9c52ec56b535cffec3bc844254c073888e6d4aeee464671ac97480/protobuf-3.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0
Using cached https://files.pythonhosted.org/packages/7b/cb/786dc53d93494784935a62947643b48250b84a882474e714f9af5e1a1928/google_auth-1.7.1-py2.py3-none-any.whl
Collecting pytz
Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting grpcio<2.0dev,>=1.8.2; extra == "grpc"
Using cached https://files.pythonhosted.org/packages/0c/47/35cc9f6fd43f8e5ed74fcc6dd8a0cb2e89c118dd3ef7a8ff25e65bf0909f/grpcio-1.25.0-cp27-cp27mu-manylinux2010_x86_64.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/93/5d/c4f950891251e478929036ca07b22f0b10324460c1d0a4434c584481db51/cffi-1.13.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3"
Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting rsa<4.1,>=3.1.4
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools<3.2,>=2.0.0
Using cached https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1
Using cached https://files.pythonhosted.org/packages/52/50/bb4cefca37da63a0c52218ba2cb1b1c36110d84dcbae8aa48cd67c5e95c2/pyasn1_modules-0.2.7-py2.py3-none-any.whl
Processing /home/jenkins/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3
Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, urllib3, certifi, chardet, idna, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, protobuf, googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, pytz, grpcio, google-api-core, google-cloud-monitoring
Successfully installed MarkupSafe-1.1.1 PyYAML-3.13 absl-py-0.8.1 blinker-1.4 cachetools-3.1.1 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.8 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.3.0 google-api-core-1.14.3 google-auth-1.7.1 google-cloud-monitoring-0.33.0 googleapis-common-protos-1.6.0 grpcio-1.25.0 idna-2.8 ipaddress-1.0.23 jinja2-2.10.3 ntlm-auth-1.4.0 numpy-1.16.4 pint-0.9 protobuf-3.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.7 pycparser-2.19 pytz-2019.3 pywinrm-0.4.1 requests-2.22.0 requests-ntlm-1.1.0 rsa-4.0 six-1.13.0 timeout-decorator-0.4.1 urllib3-1.25.7 xmltodict-0.12.0
[beam_PerformanceTests_WordCountIT_Py37] $ /bin/bash -xe /tmp/jenkins6373272794089139210.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py37_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py37 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE'
2019-11-20 19:47:02,914 d782f7c5 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d782f7c5/pkb.log>
2019-11-20 19:47:02,914 d782f7c5 MainThread INFO PerfKitBenchmarker version: v1.12.0-1532-g2784642
2019-11-20 19:47:02,915 d782f7c5 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=1200
--beam_it_module=:sdks:python:test-suites:dataflow:py37
--beam_sdk=python
--k8s_get_wait_interval=10
--bigquery_table=beam_performance.wordcount_py37_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE
--beam_prebuilt
--dpb_service_zone=fake_zone
--project=apache-beam-testing
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/src>
--beam_runner=TestDataflowRunner
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
--beam_python_sdk_location=build/apache-beam.tar.gz
2019-11-20 19:47:03,484 d782f7c5 MainThread INFO Setting --max_concurrent_threads=200.
2019-11-20 19:47:03,510 d782f7c5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2019-11-20 19:47:03,566 d782f7c5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2019-11-20 19:47:03,571 d782f7c5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2019-11-20 19:47:03,571 d782f7c5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2019-11-20 19:47:03,572 d782f7c5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 19:47:03,572 d782f7c5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2019-11-20 19:47:03,576 d782f7c5 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 995, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 846, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 689, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 161, in Run
job_type=job_type)
TypeError: SubmitJob() takes at least 3 arguments (5 given)
2019-11-20 19:47:03,576 d782f7c5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2019-11-20 19:47:03,577 d782f7c5 MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-11-20 19:47:03,577 d782f7c5 MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d782f7c5/pkb.log>
2019-11-20 19:47:03,577 d782f7c5 MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py37/ws/runs/d782f7c5/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org